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CHAPTER  1 


INTRODUCTION 


1.1  The  Conceptual  Framework 

The  purpose  of  this  investigation  is  to  study 
organizations,  their  structure,  and  their  development  from 
the  viewpoint  of  general  systems  theory.  In  general,  organi¬ 
zational  structure  is  characterized  by 

1.  A  division  of  responsibility  among  organiza¬ 
tional  subunits 

2.  The  subunits'  means  of  exercising  these 
responsibilities 

3«  Measures  of  the  subunits'  performances 

lj.  Means  of  signalling  between  subunits  to 
indicate  actions  to  be  taken. 

The  responsibility  which  is  divided  is  one  of  controlling 

a  process  so  that  some  useful  task  is  accomplished  as  the 

process  unfolds.  Item  2,  then,  includes  the  manner  in  which 

control  actions  are  transmitted  to  effect  the  evolution  of 

the  process. 


The  theory  of  self -organizing  systems  is  con¬ 
cerned  with  systems  which  are  capable  of  modifying  their  own 
structure.  This  theory  can  be  of  value  in  operations  research 
by  supplying  measures  of  organizational  efficiency  and 
mathematical  models  of  changes  in  the  structure  of  organi- 
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zations,  Although  reorganization  has  been  recognized^ ^  as 
a  common  method  of  attacking  organizational  inefficiency,  the 
study  of  structural  changes  in  organizations,  i,e,  *•  organization 
theory,**  is  mostly  qualitative  in  character,)  An  effective 
theory  of  self -organizing  systems  would  contribute  substan¬ 
tially  towards  quantifying  organization  theory. 

Two  different  types  of  self-organizing  systems 
are  discussed  by  Mesarovic, causal  and  teleological. 

Causal  self-organizing  systems  are  "preprogrammed"  to  change 
their  structure  in  response  to  certain  stimuli.  Teleological 
self -organizing  systems,  on  the  other  hand,  have  a  specified 
goal  or  purposej  the  structure  modifications  are  carried  out 
by  the  system  itself  so  as  to  pursue  this  goal  as  effectively 
as  possible.  The  behavior  of  a  teleological  self -organizing 
system  is  ourDOseful!  that  is,  it  involves  a  goal-oriented 
choice  of  a  particular  structure  from  a  set  of  possible  struc¬ 
tures,  Since  operations  research  is  concerned  with  purposeful 
systems,  we  will  confine  our  study  to  teleological  self¬ 
organizing  systems. 

One  of  the  difficulties  encountered  in  the  study 
of  such  systems  is  the  absence  of  an  ordering  relation  defined 
on  the  set  of  all  possible  structures  of  the  system.  Since 
organizations  are  themselves  teleological  self -organizing 
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systems,  this  difficulty  may  have  contributed  to  the  sparsity 
of  quantification  in  organization  theory.  The  main  task  of 
this  thesis  is  to  develop  such  an  ordering  relation,  so  that 
the  system  will  have  a  basis  for  determining  the  "best**  struc¬ 
ture  under  which  to  operate. 

The  conceptual  framework  within  which  we  will 
study  teleological  self -organizing  systems  is  that  of  a  "multi¬ 
level,  multi-goal  (mLnG)*  system.  In  the  mLnG  renresentation 
considered  in  this  thesis,  the  researcher  has  complete  know¬ 
ledges  of  the  components  and  their  interactions.  These  compon¬ 
ents  are 

(1)  transformation  elements 

(2)  goal-seeking  elements. 

Elements  of  type  (1)  specify  a  set  of  transitions  on  a  set  of 

(<) 

operands  into  a  set  of  transforms. v  '  For  example,  the  trans¬ 
formation  T(a— >b,  b— ^c,  c-)a),  effects  the  transition 
cab-^abc,  from  the  operand  "cab"  to  the  transform  "abc." 

Consider  a  set  S  of  transformation  elements, 
each  member  of  S  acting  on  the  same  set  of  operands.  Elements 
of  type  (2)  are  assumed  to  have  two  capabilities? 


•*As  opposed  to  the  "black  box  problem,"  where  an 
experimenter  attempts  to  deduce  the  contents  of  some  unknown 
system  by  adjustment  and  measurement  of  its  inputs  and  out¬ 
puts,  respectively. 


k 

(i)  selection  of  a  particular  member  of  S 

(ii)  "influencing'*  another  goal-seeking  element 
in  its  choice  of  a  particular  member  of  S«* 

Also,  goal-seeking  elements  have  a  purpose  attributed  to  them. 
The  degree  of  success  of  achievement  of  this  purpose  depends 
on  the  transforms  of  the  operands  resulting  from  the  trans¬ 
formation  selected.  The  reader  is  reminded  that  we  are  not 
concerned  with  detecting  the  purpose;  the  assumption  will  be 
made  that  a  well-defined  purpose  exists  for  each  goal-seeking 
element  in  the  system, 

1.2  Scope  and  Outline  of  the  Investigation 

In  the  sequel,  we  will  be  concerned  with  the 
situation  Yrtaere  the  responsibility  for  choosing  a  transforma¬ 
tion  from  the  set  S  is  divided  among  more  than  one  goal- 


■»As  an  example,  let  S  consist  of  the  two  trans¬ 
formations  T^(a-^b,  b— >c,  c— >a)  and  T2(a~*c>  b-^b,  c-4a), 

and. the  system  be  composed  of  S  and  a  goal-seeking  element 
G^  »  If  the  purpose  of  the  goal-seeking  element  G-^  is  to 

achieve  the  result  "acb"  from  the  operand  "cab;"  clearly  T2  is 

the  transformation  it  should  choose.  If  we  introduce  another 
goal-seeking  element  G2  into  the  system,  having  the  capability 

to  alter  G-^' s  goals  and  with  the  purpose  of  achieving  the 

result  "abaca"  from  the  operand  "cacbc,"  then  G2  should  change 

G^'s  desired  result  to  "abc"  in  order  to  cause  G-^  to  select 
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seeking  element  -with  the  capability  of  type  (i)  noted  above. 

The  details  of  how  this  is  contrived  are  deferred  until  the 
next  chapter^  there,  we  will  construct  our  system  so  that  the 
contribution  which  each  goal-seeking  element  makes  towards 
selection  of  a  particular  member  of  S  affects  the  degree  of 
achievement  of  the  goals  of  all  the  elements  responsible  for 
the  choice.  In  addition,  we  will  be  interested  in  hierarchical 
arrangements  of  goal-seeking  elements,  where  the  group  of 
goal-seeking  elements  with  the  divided  responsibility  described 
above  are  influenced  by  goal-seeking  elements  having  the 
capability  (ii)  noted  earlier. 

The  popular  examole  of  the  thermostat -furnace 
system  serves  well  to  illustrate  the  concept  of  a  multi¬ 
level,  multi-goal  system,  as  well  as  how  the  goals  of  the 
individual  elements  will  be  assumed  to  affect  each  other.  TTe 
imagine  a  large  apartment  house,  where  each  individual  apart¬ 
ment  has  its  own  thermostat-controlled  furnace.  The  apartment 
house  is  single-storied  and  the  walls  between  apartments  are 
quite  thin,  so  that  the  thermal  diffusivity  between  apartments 
is  high  enough  for  the  temperature  in  each  apartment  to  be 
affected  by  the  temperatures  in  adjacent  apartments.  Each 
apartment  dweller  knows  none  of  his  neighbors,  so  that  these 
heat  sources  or  sinks  are  lumped  with  the  environmental  dis- 


6 


turbances,  i,e»  outside  air  temperature]  thus,  there  are  no 
abnormal  behavioral  consequences  present,  such  as  an  apartment 
dweller  being  willing  to  undergo  extreme  discomfort  in  order 
to  inflict  an  unusual  disturbance  on  his  neighbor  % 


In  each  apartment,  the  thermostat  can  be  thought 
of  as  choosing  an  element  from  the  set  of  transformations 
composed  of  the  two  elements, 

1.  cold  air - ?  cold  air, 

2.  cold  air - ^-hot  air, 

in  order  to  achieve  its  goal  of  maintaining  a  level  of  tem¬ 
perature  in  a  room.  The  set  of  transformations  from  the 
over-all  viewpoint,  that  is,  for  the  system  composed  of  all 


the  thermostats  and  furnaces  in  the  building,  is 

cold  air - *Y^  by  thermostat  No,  1  | 

J  cold  air - by  thermostat  No,  2  [ 


cold  air — >Y„  by  thermostat  Ho,  N 

N 


i  =  1,2,3, . .  .,2  , 


where  the  Y^'s  assume  all  combinations  of  the  values  "cold 

j 

air”  or  "hot  air"  as  the  subscript  i  ranges  over  its  indi¬ 


cated  values.  This  is  an  example  of  1LNG  control. 


The  occupants  of  these  apartments  control  their 
individual  furnaces  from  the  second  level,  by  setting  the 
goals  of  the  thermostats.  The  system  composed  of  the  furnaces, 
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thermostats,  and  apartment  dwellers  is  a  2L2JG  system.  If  the 
apartment  house  has  a  "house  physician,**  who  prescribes  a 
different  room  temperature  to  each  tenant  (the  goal  of  the 
house  physician  being  to  keep  the  tenants  as  healthy  as 
possible),  inclusion  of  this  individual  defines  a  3L(2N+1)G 
system.  The  physician  indirectly  exerts  control  on  all  the 
furnaces  down  through  the  tenant-thermostat  hierarchy. 

The  idea  of  a  "state  of  equilibrium*  will  be  needed 
in  the  sequel.  The  systems  which  we  will  study  in  this  and 
succeeding  chapters  will  be  dynamic  systems,  characterized  at 
a  particular  time  t  by  the  values  of  a  finite  set  of  numeri¬ 
cal  quantities  x^(t),  Xg(t),  ...,  x5(t).  These  quantities, 
called  "state  variables,"  constitute  the  components  of  a  vector 
x(t),  the  state  vector.*  The  "line  of  behavior"  of  a  dynamic 
system  is  a  trajectory  in  s-dimensicnal  space,  governed  by 
differential  equations  if  changes  occur  continuously  in  time, 
or  by  difference  equations  if  changes  occur  at  discrete  times, 

t  ,  t  4  At,  ...,  t  4*  nAt,  ....  In  the  continuous  case,  if 
0  0  0 

the  state  vector  remains  constant  over  a  non-zero  interval  of 
time,  however  small,  that  state  is  a  state  of  equilibirum. 

In  the  discrete  case,  a  state  of  equilibrium  is  characterized 


*An  underlined  quantity  denotes  a  vector  through¬ 
out  this  thesis. 


8 


by  the  equation  x(t^+/4t)  =  x(t^)  .  In  either  case,  a  state 
of  equilibrium  demonstrates  the  property  of  being  unchanging 
in  time.  This  investigation  Trill  be  confined  to  the  discrete 
case. 


Notice  that,  because  of  the  way  we  look  at  the 
system,  i.e.  microscopically,  when  the  entire  system  is  in  a 
state  of  equilibrium,  each  element  composing  the  system  is 
also  in  a  state  of  equilibrium.  That  is,  each  part  of  the 
system  is  in  a  state  of  equilibrium  in  the  conditions  provided 
by  the  other  oarts.  One  can  also  demonstrate  the  converse 
statement  as  Ashby^  does,  in  order  to  arrive  at  the  result: 
the  whole  system  is  at  a  state  of  equilibrium  if  and  only  if 
each  part  is  at  a  state  of  equilibrium  in  the  conditions  pro¬ 
vided  by  the  other  parts. 

Control  of  a  discrete  dynamic  system  is  a  matter 
of  selecting  a  transformation  element  from  the  set  of  such 
elements,  i.e0  exerting  a  "control  action,"  at  each  transition 
so  that  the  behavior  is  in  some  sense  best.  In  the  mLnG 
approach  considered  here,  a  performance  measure  is  associated 
with  each  goal-seeking  element. 

The  mLnQ  systems  considered  in  this  investigation 
have  a  single  highest-level  or  "apex"  goal-seeking  element; 
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its  goal  is  considered  to  be  the  over-all  system  goal.  This 
element  will  always  have  the  capability  of  type  (ii).*  We 
will  assume  the  apex  unit  has  all  the  information  to  determine 
the  over-all  optimal  control  law,  but  because  of  its  capabilities, 
must  influence  other  goal-seeking  elements  to  implement  it. 


The  position  of  a  structure  in  the  ranking  of  a 
structure  set  under  the  ordering  relation  (mentioned  in  the 
previous  section)  to  be  developed  is  determined  by  the  effi¬ 
ciency  of  the  lower  level  units  in  synthesizing  the  optimal 
control  law  when  operating  in  that  structure ,  In  order  to 
illustrate  this,  assume  that  in  a  mlnG  system; 

1,  The  lower  level  goal-seeking  elements  are 
collectively  exerting  control  by  engaging  in 

a  temporal  "acti on-counter action" type  of 
interplay  in  an  attempt  to  arrive  at  a  state 
of  equilibrium,  where  the  action  of  each  unit 
is  best  under  the  conditions  imposed  by  the 
other  units, 

2.  The  goal-seeking  element  at  the  apex  has  the 
ability  to  inf luence-H-*  these  lower-level  ele¬ 
ments  so  that  their  collective  equilibrium 
control  actions  coincide  with  the  optimal 
control  rule  from  the  over-all  viewpoint. 


mLnG  systems  with  this  characteristic  have  been 
termed  ''indirect  intervention"  systems  by  Mesarovi.c"« (17>18) 

In  order  to  maintain  autonomy  of  the  lower-level 
goal-seeking  elements,  this  will  not  be  in  the  form  of  a 
"directive"  as  to  what  control  action  to  apply. 
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3«  The  rate  at  which  the  interplay  described  in 
(1)  above  approaches  equilibrium  is  at  least 
partially  determined  by  the  structure  under 
which  the  lower-level  goal-seeking  elements 
operate , 

As  long  as  the  interplay  described  in  (1)  is  in  the  '•transient 
phase,"  because  of  (2),  the  over-all  optimal  control  law  is 
not  being  achieved*  Because  of  (3),  we  can  rank  the  struc¬ 
tures  according  to  how  well  this  control  law  is  approximated; 
a  comparison  of  two  structures  would  lead  to  a  designation  of 
the  one  with  the  higher  rate  of  convergence  as  the  better* 

For,  this  would  assure  arrival  at  equilibrium,  and  hence 
optimal  control,  in  the  smaller  amount  of  time* 

Suppose  we  allow  the  highest  level  goal-seeking 
element  in  a  mLnG  system  the  additional  (to  the  type  (ii)  cap¬ 
ability  already  assumed)  capability  to  change  the  existing 
structural  arrangement  of  the  lower  level  elements.  With  this 
capability  and  the  method  of  ranking  structures  described 
above,  this  system  displays  behavior  which  we  would  classify 
as  "self-organizing «*  The  goal-seeking  element  at  the  highest 
level  can  alter  the  structure  so  as  to  "select"  a  structure 
from  a  set  of  different  possible  structures  "below"  it.  This 
behavior  is  also  purposeful,  in  that  the  criteria  of  choice 
assures  the  best  approximation  to  the  optimal  control  law* 
i,e,  is  "goal-oriented";  thus,  under  the  assumptions  made  in 
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the  previous  paragraph,  we  would  have  a  teleological  self- 
organizing  system. 

In  the  next  chapter  we  will  formulate  a  general 
mathematical  framework  within  which  we  shall  quantify  some 
of  the  ideas  discussed  above.  Chapter  IT  also  gives  defini¬ 
tions  of  the  concepts  which  will  be  employed  in  this  thesis. 

Chapters  III  through  VI  develop  the  ideas  dis- 
cussed  above  for  an  important  special  case,  a  linear  system 
with  a  quadratic  loss  function.  In  Chapter  HI  the  optimal 
control  law  is  derived.  Chapter  IV  is  concerned  with  the 
action-counteraction  interplay  between  two  goal-seeking  ele¬ 
ments  and  Chapter  V  discusses  how  a  higher-level  unit  uses  its 
influence  to  effect  optimal  control  out  of  this  interplay. 
Chapter  VI  is  concerned  with  developing  a  self -organizing 
system  in  the  manner  indicated  in  some  earlier  remarks  of 
this  chapter. 


Chapter  VII  gives  the  summary  and  conclusions. 


since  it  lends  itself  easily  to  analytic  treat¬ 


ment 


CHAPTER  II 


MATHEMATICAL  REPRESENTATION  AND  DEFINITION 

2>1  Introduction 

In  this  chapter  a  mathematical  representation  of 
an  abstract  multi-level,  multi-goal  (mLnG)  system  is  con¬ 
structed.  As  we  mentioned  in  Chapter  I,  these  systems  are 
composed  of  two  types  of  elements,  transformation  elements 
and  goal-seeking  elements;  thus,  mathematical  attributes  and 
relationships  will  be  imputed  to  these  two  types  of  elements. 

The  mathematical  arguments  throughout  this  and 
later  chapters  will  be  treated  so  that  the  rigorous  aspects, 
such  as  existence  of  solutions,  interchange  of  limits,  etc., 
will  be  omitted. 

This  chapter  also  serves  to  define  some  of  the 
concepts  and  specify  some  of  the  notation  utilized  in  later 
chapters. 

2.2  Notational  Conventions 

In  this  and  succeeding  chapters,  finite  sequences 
of  column  vectors  will  be  represented  by  the  upper  case  letter 
corresponding  to  the  lower  case  designation  of  the  elements 
of  the  sequence;  thus,  M  denotes  m(l),  m(2),  ...,  m(T), 
where  T  is  finite.  Subscripts  common  to  the  elements  of 
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such  a  sequence  Trill  be  so  indicated  by  appending  them  to  the 
upper  case  designation;  thus  denotes  x^l),  ^(2),  , ,,, 

*1.(1)  • 

We  rrill  have  occasion  to  form  column  vectors  by 
"stacking"  the  elements  of  finite  sequences  of  vectors.  This 
is  denoted  by 

z  -  [z(l)  z(2)  ...  z(T)J  . 

The  symbol  /  denotes  transposition  of  vectors  and  matrices. 

Note  that  z  above  determines  a  point  in  sT- 
dimensional  euclidean  space.  This  correspondence  between 
finite  sequences  of  vectors  and  points  in  multi -dimensional 
space  will  be. utilized  often  in  the  sequel. 

2.3  Dynamical  Representation  and  Control 

The  core  of  the  mathematical  representation  con¬ 
sists  of  a  scheme  that  explains  the  trajectory  or  temporal 
behavior  of  the  point  determined  by  the  state  vector  of  the 
system,  x(t)  =[x1(t),  x^t),  ...  xs(t)]  ,  in  s-dimensiona.1 
euclidean  space, 

Ax(t)  =  £.jx(t),  m(t),  z(t)J  ,  x(0)  =  c,  (2.1) 

where  «  A"  is  the  forward  difference  operator  in  the  discrete 
case  and  the  time  derivative  in  the  continuous  case.  The 
vector  functions  m(t)  and  ?,(t)  in  (2.1)  represent  the  control- 
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led  and  uncontrolled  "inputs,1*  respectively,  the  latter  hence¬ 
forth  being  referred  to  as  the  "disturbance."  The  discussion 
will  proceed  under  the  assumption  that  the  process  governed 
by  (2.1)  is  discrete,*  so  that  (2.1)  is  a  difference  equation, 
and  can  be  rewritten  as 

x(t+l)  =  s|x(t),  m(t+l),  _z(t+l)^J  ,  x(0)  =  c,  (2.2) 

t  =  0,  1,  2,  ...,  T-l, 

Where  T  will  be  assumed  to  be  finite.  Equation  (2.2)  will 
subsequently  be  referred  to  as  the  "causal  subsystem,"  the 
letter  S  being  used  to  remind  us  that  it  is  regarded  as  a 
system. 


An  analogous  discussion  exists  for  the  continuous 

case. 

In  some  control  problems,  the  controlled  input 
(to  S)  vector  m(t)  is  itself  an  output  of  another  system* 
m(t+l)  =  A^m(tJ,  v(t+l)],  where  v  is  the  controlled  input 
£o  A.  The  vector  v  may  have  onTy  a  single  element,  just 
as  m  could  have  in  (2,2).  This  system  can  be  integrated 
with~the  causal  subsystem  S  by  defining  a  new  state  vector 
y(t)  =  x(t),  m(t)  /.  Then,  the  vector  functions  A  and  S 
can  be  combineH  into  a  new  vector  function  T  as  follows;  — 

x(t+l)  =  S  x(t ) ,  A  £m(t), 

m(t+l)  =  A^m(t),  «7^(t+l)^ 

becomes 

y(t+l)  =T[y(t),  v(t+l), 

which  is  exactly  the  same  form  as  ( 2 . 2 ) |  thus,  the  introduc¬ 
tion  of  A  above  causes  no  change  in  the  conceptual  frame¬ 
work  at  tFTis  level  of  generality. 


v(t+lj|,  *(t+l)J 
z(t+l)], 
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In  Chapter  I  we  distinguished  between  goal-seeking 
elements  which  had  the  capabilities  to  select  a  particular 
transformation  element  from  a  set  of  such  elements  and  goal¬ 
seeking  elements  which  influenced  other  goal-seeking  elements. 
In  this  chapter,  the  first  capability,  which  was  termed  type 
(i),  takes  the  form  of  selecting  certain  controlled  input 
vectors  at  each  transition,  by  regarding  the  process  deter¬ 
mined  by  (2.2)  as  the  successive  application  of  elements  in  a 
sequence  of  transformations  on  the  initial  state  x(o)  =  c» 

The  duality  between  the  sequence  of  controlled  input  vectors 
M  and  the  sequence  of  transformations 

S[x(°),  m(l),  z(lj],  s[s[x(0),  m(l),  £(1)L  m(2),  z(2)], 
...  sfsJjS  « .  o  S  £x(0) ,  m(  1) ,  jz(l)J  ,  m( 2 ) ,  2)^ ,  . . .  ,^J ,  m(T  )  * 

z(T)J  is  well  known. 

Consider  a  goal-seeking  element  G  having  the 
capability  of  controlling  the  causal  subsystem  (2.2)  through 
selection  and  implementation  of  the  controlled-input  vector 
sequence  M»  Suppose  G's  goal  is  to  minimize  a  **loss  func¬ 
tion”  g(M>  X,  U),  where  u(t)  is  the  vector  of  parameters  in 
the  loss  function  at  the  transition  x(t-l)  — >  x(t) .  These 
parameters  are  regarded  as  uncontrolled  by  G.  The  ^control 
problem”  facing  G,  then,  is  ^determine  M  so  as  to  minimize 
g(M,  X,  TJ)  subject  to  (2.2).* 
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The  solution  of  this  problem  is  an  expression 

for  the  "optimal  policy"^ ^  or  "operation**^ ^  Mq  in  terms 

of  the  sequences  U  and  Z.  This  is  denoted  by 

(M)q  =  M(U,  55)  (2.3) 

It  is  clear  that  G  must  possess  complete  information  about 

U  and  Z  if  it  is  to  evaluate  M  correctly,  i.e,  determine 

o 

the  operation  to  be  performed. 

3.U  Organizational  Structure  and  Equilibrium  State 

Any  collection  of  goal-seeking  elements,  each  of 
which  participates  in  the  control  of  a  single  causal  sub¬ 
system,  will  henceforth  be  referred  to  as  a  "controller." 
Consider  an  organizational  structure  of  a  controller  consist¬ 
ing  of  two  first  level  goal-seeking  elements  G-q  and  G^ 
having  the  capability  of  type  (i)  and  a  single  second  level 
goal-seeking  element  Gg  having  the  capability  of  type  (ii), 
which  is  characterized  by  the  following; 


In  the  mLnG  theory,  goal-seeking  elements  are 
assumed  to  have  at  least  the  problem  solving  capabilities  of 
the  researcher. 

**In eluded  in  (2.3)  and  the  comments  following  it 
is  the  case  where  the  solution  determines  an  optimal  "feed¬ 
back  controller,"  i.e.  a,  rule  of  determining  Mq  element-by¬ 
element  as  the  process  evolves,  according  to  the  values  assumed 
by  the  state  variables,  denoted  symbolically  by 

m(t+l)  =  h£x(t),  z(t+l),  u(t+l)]« 
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1.  A  reticulation  (splitting  up)  of  the  causal 
subsystem,  into  two  causal  subsystems  and 

Sgj  by  partitioning  the  vector  function  S  and 

hence  the  state  vector  X  so  that 


X-^(t+l)  =  ®l[X(t)»  ®(t+l),.  z(t+l)^ 

C2.U) 

Xg>(t+1)  =  SgTx(t),  m(t+l),  z(t+l)J 

(2.5) 

x(t)  =  [xx(t)  x2(t)]  . 

(2.6) 

2.  The  causal  subsystem  is  placed  under  the 
cognizance  of  G-^?  i  =  1,  2,  i.e.  Is 

assumed  to  be  aware  of  the  functional  form  of 

S .  but  not  of  S . ,  for  i  /  j. 

-a  -j 

3«  The  controlled  innut  vector  is  partitioned 

®(t)  =  “goo]  (2*7) 

and  G-j^  is  given  the  capability  of  select¬ 
ing  nu(t)  at  each  transition. 

1*.  It  is  assumed  that  G-^'s  goal  is  to  minimize 
the  loss  function  X^>  TJ^  )« 

5>.  The  second  level  unit  Go  has  the  capability 

to  adjust  the  parameters  U-^  and  Ug  in  the 

first  level  units'  loss  function.  G2  is  aware 

of  the  entire  system,  i.e.  the  equation  (2.2), 
the  method  of  partitioning  S>  and  the  goals 
and  means  of  control  of  G^” and  G3.2* 

6.  It  is  assumed  that  G2,s  g°al  is  to  minimize 
the  loss  function  gg(M,  X?  U). 

Figure  2ol  is  a  schematic  diagram  of  the  effects  exerted  on 

and  within  the  2L3G  system  considered  here . 
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Figure  2.1 


U 


The  dynamics  of  the  two  subsystems  S-^  and  Sg  are 

such  that,  under  identical  inputs  and  disturbances,  the 

transition  from  x(t)  to  x(t+l)  governed  by  equations  (2.U), 

(2.5),  and  (2.6)  is  identical  to  the  transition  governed  by 

(2.2).  Equations  (2.U)  and  (2.5)  can  now  be  rewritten  as 

x±(t+l)  =  S^x^t),  m^t+l),  w±(t+l)]  (2.8) 

for  i  =  1,  2,  where  w^(t+l)  is  the  vector  of  uncontrolled 

incuts  to  S.  and  contains  the  effects  of  m.(t)  and  x.(t)  for 
i  ~ "J  “* j 

i  ^  j.  The  organizational  structure  under  consideration  here 
can  now  be  characterized  as  the  simultaneous  consideration  of 


the  three  control  oroblems 
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(i)  of  G-j^  for  i  =1,  2j  "choose  so  as  to 

minimize  g^M^,  X.^,  U.j*)  subject  to  (2.8)." 

(ii)  of  Ggj  "choose  U-j*  and  Ug*  so  that  G-^  and 
G-J2  will  together  determine  M  so  as  to 
minimize  ggCM,  X,  U)  subject  to  (2.2).** 

The  solution  to  G-^'s  problem  is  an  expression  analogous  to 

(2.3), 

(Mi)0  =  M.(U.*,  W±)  (2.9) 

The  second  level  element  G^  determines  the  optimal  control 
law  (M)  from  (2.3).  Denote  that  part  of  (M)q  corresponding 
to  the  controlled  input  variables  under  the  control  of  G^ 
by  (Mi)o«  In  order  for  the  optimal  control  law  to  be  synthe- 
sized  by  the  2L3G  controller,  we  must  have 

(Mi)0  =  (M.)0,  (2.10) 

so-,  substituting  (M1)  into  the  left  side  of  (2.9)  and  solving 


for  Ui  ,  we  get 


<Oo  -  VVV  *iJ 

as  the  solution  of  Gg's  control  problem. 


(2.11) 


Under  the  conditions  stated  above,  G^  cannot, 

in  general,  determine  (M^)q  from  (2.9),  for  is  a  function 

of  X,  for  i  f  j.  The  line  of  behavior  X.  is  affected  by  X., 
J  J  ^ 

so  that  a  functional  dependence  of  on  X^  is  established. 
Since  X^  cannot  be  determined  until  is,  we  have  arrived 
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at  an  Impasse,  In  order  to  bypass  this  difficulty,  we  pro¬ 
pose  the  iterative  process  described  below. 


The  finite  T -stage  process  governed  by  (2.2)  is 
imagined  to  be  repeated  an  indefinite  number  of  times.  This 
can  either  be  interpreted  as  a.  "real"  infinite-stage  process 
with 


x(0)  =  £  =  x(T+l)  =  x(2T+2)  =  ...  =  x(nT+n)  =  ... 
z(t)  =  z(t+nT+n),  u(t)  =  u(t+nT+n), 
for  t  =  1,  2,  ...,  T  and  n  =  0,  1,  2,  or  as  a  "fictitious” 

repetitive  simulation  of  the  finite  T-stage  process.  Suppose 
Gp  adopts  the  following  procedure  in  order  to  resolve  the 
unknown  effect  of  For  the  initial  iteration  of  the  process, 
i.e,  from  t  =  0  to  t  =  T,  G-^  uses  w^(t)  =  0  for  t  =  1,  2,  ..., 

T  in  determining  its  control  input  vector  sequence  from  (2.9). 
This  is  denoted  by  (M^)-^,  the  subscript  "1"  outside  of  the 
parenthesis  determining  the  period  over  which  this  sequence 
runs.  This  notation  will  be  extended  to  the  other  quantities 
of  interest  here;  thus,  (X^)n  means  the  sequence  of  x^(t)*s 
during  the  n —  evolution  of  the  orocess.  If  we  solve  (2.8) 
for  w^(t+l),  we  can  write  the  discrepancy  in  terms  of  the  ob¬ 
served  quantities 


w±(t+l)  (t+1),  x..(t),  m^t+l)]  , 


(2.12) 


"b  —  Oj  o  *  *  j  T-l 
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Now,  after  G-^  determines  (M^)^,  as  indicated  above,  we  can 
imagine  that  "implementation"  of  this  is  instituted  by  G1i 
actually  "feeding  in"  to  S,  as  its  share  of  the  inputs,  the 
vectors  nu(l),  nn(2),  ...,  itk(t)  at  the  appropriate  times. 

G,  .  can,  in  turn,,  observe  the  subvector  x.  (t)  of  the  state 
vector  x(t)  at  each  time  step,  and  with  this  information,  can 
compute  a  w.  (t)  for  each  integral  t  using  (2,12).  Denote 
the  sequence  thus  obtained  by  (W^)-^>  and  assume  that  G^  now 
uses  <*i)l  as  the  in  (2.9)  to  compute  (M.^*  Then,  the 
entire  procedure  can  be  repeated.  In  general,  we  can  ideal¬ 
ize  G-^  as  carrying  on  an  iterative  procedure,  where  it 
computes  (M^)n  by  the  formula,  analogous  to  (2.9), 


<%>.  = 


(2-13) 


then  uses  the  observations  it  makes  on  (X^)n  to  compute  the 
vector  sequence  according  to 

2Sin(t)  =Si[kLn('t),  3i.n(t-l),  fl^t)],  (2.1U) 


t-1,  2,  ...,T, 

which  is  obtained  from  (2.12).  If  the  process  ever  yields 
the  equality  ”  !*lW  we  would  say  that  the  process 

has  arrived  at  a  state  of  equilibrium,  i.e.  a  state  which  is 
unchanging  for  subsequent  periods  of  evolution. 


*A  superscript  will  be  used  to  indicate  vectors  of 
a  specific  sequence,  so  that  w.n(t)  denotes  a  vector  in  the 
sequence  (W^)n« 
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Notice  that  if  a  state  of  equilibrium  is  attained, 
i.e.  (*,)„  =  (1^,  then  (1^),,  "  (K,)^  and  (X^  = 
(X^)n+^*  Furthermore,  these  conditions  also  hold  if  i  is 
replaced  by  j,  iee.  G-^  and  G^  observe  the  event  "arrival 
at  equilibrium*  simultaneously.  This  must  hold,  because  the 


"residual"  effect  felt  by  G-^,  that  is,  the  difference  between 
the  observed  and  the  anticipated  values  of  w^(t),  is  due  to 


the  acts  performed  by  G-, 


all  other  effects  are  identical 


over  each  iteration. 


Unfortunately,  we  cannot  guarantee  that  every  such 
process  of  the  type  described  above  will  arrive  at  a  state  of 
equilibrium,  yje  can,  however,  establish  sufficient  conditions 
for  this.  The  argument  involves  a  technique  of  mathematical 
analysis  known  as  the  "method  of  successive  approximations* 
and  depends  heavily  on  the  notion  of  a  "contraction  mapping," 
which  we  now  consider. 

If  it  is  possible  to  find  a  transformation 
Pg.  =  f(p  )  of  finite-dimensional  euclidean  space  into  itself 
such  that  the  transforms  of  two  points  in  the  space  are 
nearer  to  one  another  than  the  original  points  were,  f  is 
called  a  "contraction  mapping. If  we  denote  the  distance 

*The  argument  generalizes  to  complete  metric  spaces. 
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between  two  points  and  p^  in  multi -dimensional  Euclidean 
space  by  d(p^,  p^))  this  can  be  expressed  in  mathematical 
terms  by  requiring  the  existence  of  a  positive  scalar  a  <•  1 
such  that 

d&(P!)»  f(P2)J  5  a  d(Pl»  (2.15) 

It  is  easy  to  see  that  successive  applications  of  a  contrac¬ 
tion  mapping,  i.e. 

P2  =  f(P1)>  P3  =  1(P2)>  •••>  Pn+i  =  f(Pn)»  •••  (2.15) 

would  result  in 

P  =  f(p)  (2.17) 

to  any  accuracy  desired.  For 

d[pn+l’  pn]  =  dCf(pn)»  f(pn-lil  5  ^  d[pl>  p2]  *  ^2*18) 

If  f  is  a  contraction  mapping  applied  successively  as  indi¬ 
cated  above, 


lim  d  fp  ,, ,  p~]  =  0, 
n— >  00  L  n+l  KnJ 

and  lim  p  =  p,  where  p  is  the  unique  solution  of  (2.17). 
n — >oo  rn  r  ' 


Suppose  the  vector  of  uncontrollable  variables 
^i(t)  has  q^  elements.  Then,  as  we  noted  in  section  2.2,  the 
vector-sequence  determines  a  point  in  q^T -dimensional 
euclidean  space.  The  sequence  of  vectors  (W^)^}  (Hj_)2J  ... 


thus  determines  a  sequence  of  points  in  euclidean  space  of 
q^T  dimensions.  Furthermore,  a.  mapping  between  successive 
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elements  of  the  sequence  of  W- ' s  is  implicit  in  equations 
(2.13)  and  (2.]lt);  for,  from  (2.13),  (M^)n  depends  on  (W  )Q  ^ 
and  (Wi)n  depends,  in  turn,  on  (M^)  •  Hence,  we  can  combine 
(2.13)  and  (2 •  lli )  to  find  the  mapping 

If  it  can  be  established  that  (2.19)  is  a  contraction  mapping, 
i.e.  that 

d{»i>„>  <!i>„+i}  =  £[C!dnj} 

where  a  is  a  constant  less  than  unity,  our  sufficient  condi¬ 
tion  for  convergence  to  equilibrium  would  be  established. 

Notice  that  we  could  have  just  as  well  investi¬ 
gated  the  mappings  between  (M^)n  and  (M^)n+^  or  between  (X^)R 
and  (Xj_)n+2_  ^or  contraction  property.  Furthermore,  if  we 
could  establish  that  a  mapping  between  equally  spaced  elements 
of  any  of  these  sequences,  say  between  (X^)r  and  (X  )n+^,  is 
a  contraction  mapping,  this  would  also  be  a  sufficient  condi¬ 
tion  for  convergence  to  equilibrium.  This  can  be  easily  seen 
by  noting  that  the  sequences 

<*i)l,  ^i^k+l*  ^i^k+l’  •** 

(X’i)2’  ^i^k+2’  ^Xi^2k+2’  *** 

(XiV  (xi)2k’  ^i^k*  *** 


all  converge  to  the  same  limit,  since  as  we  stated  above,  the 
solution  to  (2.1?)  is  unique;  that  is,  it  does  not  depend  on 
the  point  from  which  the  sequence  of  successive  mappings 
begins. 


A  slight  conceptual  difficulty  arises  in  cases 

where  the  iterative  process  converges,  i.e.  a  1  in  (2.20), 

but  an  infinite  number  of  iterations  is  required  to  obtain 

(W. )  =  (W. )  . Henceforth,  we  will  refer  to  “achievement 

i/n  '  i'n+1 

of  equilibrium  to  any  prescribed  degree  of  accuracy.**  Given 

a  6  >  0,  if  the  sequence  (VT  ) ^ converges,  there 

exists  a  finite  N  such  that  (W^)n+^j-c  6  for 

n  >  N;  thus,  in  this  case,  “achievement  of  equilibrium  within 

"bh. 

5  “  would  occur  at  the  N—  iteration. 


The  iterative  process  described  above  is  conducted 
at  the  first  level,  Suppose  a  <  1  in  (2.20)  and  let  (W^)g 
denote  the  solution  of  vr  =  f(?r),  where  f  is  the  mapping 
(2.19).  The  “control  action  in  equilibrium,'*  (M^)g,  and  (W^)g 
are  connected  by  the  relationship,  analogous  to  (2. 9), 


<«i>.  “  <Wi>e]> 


(2.21) 


(M-^)g  has  the  property  that  it  is  an  optimal  policy  for  G^i5 
in  the  conditions  IT-j*"  imposed  by  G2>  'when  G-^  uses  the  poliqy 
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Returning  to  the  consideration  of  G2*s  control 
problem,  it  is  seen  that  (2.11)  contains  just  as  (2.9) 
does.  Since  G2  is  aware  of  the  entire  system  and  the  optimal 
control  law  (M)  it  can  obtain  (W^)Q  and  (T^)^  ^7  s°iving 

(2.12)  for  i  =  1,  2,  and  for  t  =  1,  2,  ...,  T  when  the  opti¬ 
mal  control  law  is  applied.  The  following  expression, 

o»i\  = 

obtained  from  (2.11),  will  assure  that  the  first  level  itera¬ 
tive  procedure  converges  to  the  optimal  control  law,  i.e. 
that  (Mi)o  =  (M.)e, 

If  it  is  difficult  to  obtain  (W^)Q  hy  solving 

(2.12)  as  indicated  above,  G2  may  "assume**  an  equilibrium 

value  for  say  (W^)^;:  then,  using  (W^)1  in  (2.11),  obtain 

♦  X 

(U^  )  .  The  iterative  procedure  between  the  first-level 

elements,  assuming  it  converges,  arrives  at  the  equilibrium 

1  *1 
value  (W. )  ,  under  the  conditions  imposed  by  (U-,  )  and 

X  6  -i. 

*  1  2  1 

(Ug  )  «  G2  now  designates  (W.^)  =  (ll^)0  and  repeats  the 

process  for  (W^)  to  determine  a  new  first-level  value  (W^)e  • 
Continuation  of  this  process  defines  a  "nested”  iterative  pro¬ 
cedure,  where  each  single  step  in  the  (W^)n  iteration  involves 
the  entire  (*i)n  iteration  described  earlier.  Convergence 


*Within  some  prescribed  degree’  of  accuracy. 
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requires  that 

(Wi)n+1  =  h[(Wi)n]  (2.23) 

be  a  contraction  mapping,  as  well  as  f  in  (2.19).  This 
case  will  not  be  considered  further  in  this  chapter.  It 
will  apnear  again  in  Chapter  VII. 

It  is  interesting  to  examine  the  relation  between 
the  concept  of  equilibrium,  as  we  use  it  here,  to  similar 
ideas  in  other  theories..  In  mechanics,  the  idea  of  equili¬ 
brium  plays  a  large  role,  particularly  in  the  sub-area  called 
"statics'*.  A  weigit,  hanging  by  a  chain,  is  in  equilibrium, 
the  downward  force  exerted  by  gravity  being  exactly  balanced 
by  the  upward  force  of  the  chain.  In  ''dynamics",  the  concept 
is  also  imoortant,  and  begins  to  resemble  our  ideas  above. 

If  we  imagine  the  weight  to  be  Dushed  so  that  s  swinging 
motion  is  introduced,  the  forces  of  friction  impose  a  contrac¬ 
tion  mapping  on  this  system,  in  that  the  extreme  point  of  the 
arc  of  each  swing  is  closer  to  the  "point  of  rest",  or 
"equilibrium  point"  in  our  notion,  than  the  extreme  point  of 
the  arc  of  the  previous  oscillation.  This  is  an  example  of 
.  "Lyapunov  stability. 

Game  theory  also  uses  the  concept  of  "equilibrium 
point" in  a  manner  quite  similar  to  that  described 
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above.  The  iterative  process  described  earlier  can  be  thought 

of  as  either  a  temporal  repetition  of  a  “game  of  prescribed 

duration” or  as  the  “fictitious  play*^^  of  such  a  game. 

Game  theory  also  utilizes  the  idea  of  “mappings  with  a  single 
Mo) 

fixed  point*'  '  to  determine  points  of  equilibrium.  A 
particular  application  of  game  theory  to  the  types  of  problems 
arising  in  multi-level  control  is  discussed  in  Chapter  IV. 


Various  theories  in  economics  utilize  the  idea  of 

equilibrium  to  explain  prices  based  on  laws  of  supply  and 

(21) 

demand.  The  “dynamic  cobweb*'  '  concept  is  based  on  the  fact 
that  the  present  supply  creates  prices  causing  a  specific 
demand  which,  in  turn,  leads  to  a  new  level  of  supply.  This 
procedure  is  somewhat  analogous  to  the  iterative  process  de¬ 
scribed  earlier  in  that  it  defines  a  contraction  mapping 
under  certain  conditions.  Notice  in  Figure  2.2  that  the  dis¬ 
tance  between  S2  and  is  less  than  the  distance  between 
and  Sg. 


Figure  2.2 


Price 


Quantity 
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As  we  stated  in  Chapter  I,  we  propose  the  rate  of 
convergence  to  equilibrium  as  a  means  of  comparing  the  effect¬ 
iveness  of  different  structures  of  a  mLnG  system.  Referring 
back  to  the  characterization  of  a  structure  made  at  the  be¬ 
ginning  of  this  section,  it  is  clear  that  we  can  represent 
each  element  of  the  structure  set  considered  here  by  the  two 
vectors  ^  =  (a~v  o-g)  and  y,  =  (p^,  p2,  pr), 

called  "selector  vectors",  having  the  following  properties: 

<r~.  -  i  if  x.(t)  is  under  the  cognizance  of  G^ . , 
where  i  =  1,  2,  j  =  1,  2,  ...,  s,  for  the  special  case 
above,  and 

pk  =  i  if  nu(t)  is  manipulated  by  G-^ 
where  k  =  1,  2,  r  and  again  i  =  1,  2.  For  example, 
with  five  state  variables  and  6  controllable  variables 
cr  =  (1  2  112) 
t  =  (2  2  1  2  1  1) 
would  mean 

*l(t)  =  [xx(t)  x3(t)  xu(t)]  ",  X2(t)^2(t)  x5(t)]  ' 

and 

H^Ct)  =  [m^t)  m2(t)  m^(t)]  m2(t)  =  [m^t)  m^(t)  m6(t)J 

If  the  rate  of  convergence  of  the  iterative  process 
between  the  two  first  level  goal-seeking  elements  varies  as 
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the  structure  changes,  then  that  structure  which  affords  the 

most  rapid  rate  of  convergence  is  best.  For,  suppose  the 

repetition  of  the  finite-stage  process  is  regarded  as  an  in- 

finite-stage  periodic  process,  as  mentioned  earlier.  The 

faster  the  rate  of  convergence,  the  better  the  approximation 

of  the  sequence  (M)^  (M)2>  Wy  ...  to  the  optimal  control 

law  for  this  process,  (M)Q,  (M)q,'  (M)q,  ...  .  Here,  (M)n 

denotes  the  combined  control  actions  (M, )  and  (M~)  for  the 

'  l'n  £  'n 

n—  "period",  and  lim  (M)  =  (M)  ,  through  G„*s  "influence" 

n — >oo  x  'n  '  'o  2 

as  noted  earlier. 

2«5>  Self -Organizational  Activity 

In  this  section  allowed  the  additional 

capability  of  changing  the  organizational  structure  of  the 
controller.  An  example  of  a  "structure  change"  as  it  is  re¬ 
garded  in  this  investigation  would  be  "take  x^(t)  from  G^’s 
cognizance  and  place  it  in  G^’s,  i«e«  change  cr’j{  from  2  to  1, 
making  appropriate  changes  in  the  loss  function,  the  reticula¬ 
tion  specified  by  (2.U ) ,  (2.5),  and  (2.6),  and  the  means  of 
influencing  and  G^."  The  contraction  factor  a  in 
(2.20)  could  be  regarded  as  a  structural  parameter,  since  it 
decreases  as  the  rate  of  convergence  increases}  thus,  if  the 

*To  the  capability  of  type  ii  already  granted  it. 
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above  change  produces  a  reduction  in  a,  it  would  be  classi¬ 
fied  as  a  change  for  the  good.  The  variation  of  the  parameter 
a,  then,  establishes  an  ordering  relation  over  the  set  of 
structures  considered  here. 

The  soecification  of  and  by  according 
to  (2.22)  can  be  regarded  as  a  mathematical  convenience.  It 
allows  us  to  hold  the  equilibrium  point  constant  while  varying 
the  structure.  This  results  in  a  concentration  on  the  self- 
organizational  aspects,  the  main  objective  of  this  thesis. 

Let  us  regard  the  iterative  process  as  determining 
a  dynamic  system  whose  state  trajectory  is  X-^,  X,,,  ...  , 

It  is  interesting  to  compare  the  problem  of  which  structure 

(9) 

should  choose  with  a  conventional  control  problem v 
"given  a  dynamical  system 

y(t+l)  =p[y(t),  f(t)], 

find  the  vector  function  f(t)  such  that  given  an  initial  value 
y(0),  the  system  reaches  equilibrium  in  the  shortest  possible 
time."  The  "structure-choice  oroblem"  is  strikingly  similar] 
"given  a  dynamical  system 

where  the  transformation  Q  depends  on  the  structure  and  has 
a  contraction  factor  a,  find  the  structure  such  that  given  an 
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initial  value  (Dv  the  system  reaches  equilibrium  in  the 
shortest  possible  time.” 

Throughout  this  chapter  the  discussion  has  been 
held  to  a  high  degree  of  generality.  Such  things  as  the 
question  of  the  stability  of  equation  (2.U)  under  the  ”forcing 
function”  represented  by  M  and  the  ”realizability”  of  the 
solution  M  have  been  assumed  away.  In  the  remainder  of 
this  thesis,  we  will  investigate  special  cases  which,  although 
simple,  will  contain  all  of  the  features  brought  out  here. 

In  particular,  Chapter  VI  is  concerned  with  the  application 
to  a  specific  case  of  the  general  statement  made  above,  namely 
that  the  rate  at  which  a  reticulated  mLnG  system  approaches 
equilibrium  is  a  good  measure  of  the  effectiveness  of  the 


reticulation 


CHAPTER  III 


OPTIMAL  CONTROL  FOR  A  IL1G  SYSTEM 

3«1  Introduction 

The  purpose  of  this  chapter  is  to  describe  a 
particular  control  problem  which  we  will  be  concerned  with 
in  the  sequel.  The  most  important  feature  of  this  control 
problem  is  that  it  is  solvable  analytically  using  standard 
techniques.  Although  the  problem  is  simple,  it  still  con¬ 
tains  all  the  relevant  components  of  the  general  mathematical 
representation  stated  in  the  previous  chapter.  The  solution 
will  be  derived  under  the  assumption  that  the  controller  is 
of  the  simplest  possible  structure,  i.e.  single-level 
single-goal,  since  this  is  the  form  in  which  it  mil  be 
applied  later.  Of  particular  interest  will  be  the  formulas 
expressing  the  operation  to  be  performed  by  this  single  goal 
unit  in  terms  of  the  state  variables,  uncontrollable  vari¬ 
ables,  and  disturbances;  we  will  use  these  many  times  in 
what  follows. 


The  notational  convention  of  representing 
sequences  of  vectors  by  upper  case  letters  is  carried  on  in 
this  chapter. 


The  situation  we  will  consider  here  is  this*  a 
system  s  is  such  that  changes  in  its  state  vector  x(t)  occur 
at  discrete  instances  of  time.  These  changes  can  be  described 
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by  the  linear,  dynamic,  vector-matrix  difference  equation 
x(t+l)  =  Ax(t)  +  z(t+l)  +  m(t+l),  (3.1) 

over  a  finite  number  of  time  periods,  t  =  0,  1,  2,  . ..,  T-l, 
with  initial  conditions  x(0)  =  c.  The  vector  ^z(t)  is  consider¬ 
ed  to  be  an  uncontrolled  incut  or  disturbance  which  is  pre¬ 
dictable  without  error  over  the  entire  time  domain,  hence 
is  essentially  a  vector  of  parameters.  The  vector  m(t)  is 
the  controlled  input  vector,  having  the  same  number  of 
elements  as  x(t).  The  matrix  A  is  non-singular  and  constant 
over  the  period  of  interest. 

A  goal  unit  G  is  charged  with  guiding  the  state 
trajectory  x(l),  x(2),  ...,  x(T)  of  S  along  a  path  which 
minimizes 

g(M,X)  =  ^  [x(t)  -  u(t)J'  [xft)  -  u(t)J  (3.2) 

t=l 

T 

+  m*  (t)  Dm(t), 

t=l 

where  D  is  a  positive-definite  diagonal  matrix  and  U  is 
a  known  vector  sequence,  with  the  vectors  u(t)  having  the 
same  number  of  elements  as  x(t),  and  with  a  definite  element- 
by-element  association  between  the  two.  The  seqence  U 
can  be  thought  of  as  determining  an  "ideal  tra  jectoiy*  along 


which  G  '‘desires**  the  actual  state  trajectory  determined  by 
X  to  move.  For- example,  if  x^(t)  is  "steel  capacity  at  t,« 
u^(t)  would  be  "desired  steel  capacity  at  t.«  The  perfor¬ 
mance  criteria  g(M,X),  then,  represents  a  balance  between 
the  cost  of  tolerating  nonideal  behavior  and  the  cost  of 
doing  something  about  it. 

The  set  of  acts  available  to  G  at  any  particular 
transition,  say  at  the  transition 
x(t-l). - »x(t), 

is  simply  the  numerical  adjustments  m  (t),  m  (t),  ...,  m  (t) 
made  on  the  corresponding  elements  of  the  vector  Ax(t-l)  + 
zjt).  The  "operation**  or  "optimal  policy"  performed  by  G 
is  emission  of  the  vector-sequence  of  these  adjustments,  M» 
which  minimizes  (3*2).  The  operation  is  assumed  to  be 
realizable.  We  now  proceed  to  find  the  operation,  i.e.  solu¬ 
tion  to  the  minimization  problem. 

Notice  that  the  entire  process  governed  by  (3*1) 
over  the  times  0,  1,  2,  . ..,  T,  can  be  written  as  a  "closed 
form"  solution  of  the  difference  equation  (3*1)>  in  terms  of 
the  initial  states  and  sub  sequent  control  input  vectors  and 
disturbances.  This  is  obtained  by  applying  the  formula,  de¬ 
rived  in  Appendix  A, 
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This  can  be  further  abbreviated  in  vector-matrix  form: 

X  =  k[m  +  z]  +  c  ,  (3.3) 


where  K  = 


I  0  0  ...  0 
A  I  0  ...  0 

I  ...  0 


A2  A 


.T-l  .T-2 

A  A  0  0  0  A  X 


,  a  matrix  with  matrix 


elements,  and 


C  = 


A  c 

a2; 


T 

Ac 


The  "loss  function"  now  becomes 

g(M,X)  =  [x  -  u]'  [X  -  U,]  +  M'EM 


where 


D  0 

0  D 


0  ...  0 

0  ...  0 


•  «  •  « 


3»2  Derivation  of  the  Optimal  Control  Rule 

The  simple  nature  of  this  particular  control 
problem*  is  now  evident •  The  problem  is  "minimize 


*This  is  simply  the  sT -dimensional  analogue  of 
the  scalar  problem,  "minimize  (x-u)^  +  enr  ,  subject  to 
x  =  k(m+z)  +  c,  where  c  >  0." 
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(X  -  U)»  (X  -  U)  +  M'EM,  subject  to  X  =  K(M  +  2)  +  C ,  where 
E  is  a  do  sit  ive-de  finite  matrix.''  It  can  be  solved  either  by- 
employing  the  method  of  Lagrange's  multipliers,  or  by  substi-r 
tution  of  K(M  +  Z)  +  C  for  X  directly  in  g;  we  will  do  the 
latter.  Let  g^M)  be  used  to  denote  that  this  substitution 
has  been  made; 

g(M)  =  [K(M  +  Z)  +  C  -  fj'  •  [K(M  +  Z )  +  C  -  uj  (3.4) 
+  M'EM 

=  M'(K'K  +  E)M  +  2(KZ  +  C  -  U)'KM 
+  (KZ  +  C  -  U)' (KZ  +  C  -  U). 

The  diagonal  matrix  E  is  positive  definite  and, 
since  K  is  non-singular,  K'K  is  also  Dositive  definite. 

Hence,  K'K  +  E  is  positive  definite.  The  positive  definite¬ 
ness  of  K'K  +  E  is  a  necessary  and  sufficient  condition  for 
g  to  be  a  strictly  convex  function  of  the  elements  of  M. 

In  order  to  find  the  stationary  point  of  ( 3 j 
we  differentiate  g(M)  with  respect  to  the  elements  of  M  and 
solve  the  system  of  linear  algebraic  equations  which  results 
from  setting  these  derivatives  equal  to  zero? 

*The  derivative  of  the  quadratic  form,  g  with 
respect  to  the  vector  M  is  a  vector  of  the  same  dimension  as 
Me  The  details  of  this'  representation  are  relegated  to 
Appendix  B« 
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J|  =  2EM  +  2K'KM  +  2K» (KZ  +  C  -  U)  =  0  (3*3) 

==>M  =  (K'K  +  E)-1K*(U  -  C  -  K2).  (3.6) 

The  minimum  value  of  g(M)  can  now  be  found  by- 
substituting  M  as  obtained  from  (3«6)  into  g(M)>  as  given  by 
(3.U);  thus, 

(3.7) 

g  ^(M)  =  (U  -  C  -  IS)'  [i  -  K(K'K  +  E)-1K»]  (U  -  C  -  KZ). 

To  determine  the  state  vector  sequence  X  which 
results  from  "implementing’*  the  operation,  (3*6)  is  substi¬ 
tuted  into  (3.3) >  resulting  in 

X  =  K(K'K  +  E)-LK'U  +  k[t  -  (K'K  +  E)-1K'k)z 
+  [ I  -  K(K'K  +  E)_1Ki]c, 
or 

-i  r  1  (^*®) 

X  =  KCK'K  +  E)  (K'U  +  EZ)  +  [i  -  K(K'K  +  E)  TC'JC, 

3,3  Stability 

In  order  to  investigate  the  stability  of  the  above 
process,  it  is  first  necessary  to  introduce  the  concept  of 
a  "norm"  of  both  vectors  and  matrices.  Define  the  norm  of 
a  vector  as 

iiiji  -  =[r*iW>j1/2  • 

t=i 


(3.9) 
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The  norm  of  a  matrix  A  is  said  to  be  '’compatible**  with  the 
norm  of  a  vector  as  defined  above  if 

I  M  I  <  I  |A |  |  (HI.  (3.10) 

A  method  of  constructing  a  matrix  noim  so  as  to  satisfy  this 

compatibility  condition  is  to  apply  the  formula 

I  Ml  =  max  | |Ax|  |  .  (3.11) 

11*11  =  1  ", 


Norms  of  matrices  also  have  the  properties 


1 1 AB  |  |  < 

IUII  I  1 B |  |  , 

(3.12) 

1  |a+b|  |  5 

1  Ml  +  1  |b  1 1 , 

(3.13) 

and 

H 

il 

• 

(3.3ii) 

In  addition,  Halmoz^1^  and  Fadeeva^10)  prove  that 

Ml  =  [ji^A'A)]1/2  (3.15) 


and 

A  symmetric  =t>||a||  =  max  Qjjl1(A)  | ,  Mn(A)  |  J.  (3«l6) 

In  particular? 

A  positive  definite  = —-'p  1 1 A |  |  =p.^(A).  (3.17) 

In  (3.15),  (3.16),  and  (3*17)  immediately  above,  jj.^(A)  de¬ 
notes  the  i—  characteristic  root  of  A,  the  convention 

p,n  ^  £  p2  ■£  p,1  being  understood  here. 

The  process  described  by  (3*8)  will  be  termed 
'•stable"  if  for  each  e  >  0,  there  exists  a  6  >  0  such  that 


Uo 


any  trajectory  £(1),  £.(2),  ...,  ^(T)  satisfying  the  condition 
|  |x(0)  —  ( 0)||  6  satisfies  ||x(t)  -£(t)||  <  e  for 

t  =  1,  2,  ...,  T.  In  order  to  demonstrate  the  stability  of 
(3.8) >  re-write  c  as 


C  = 


A  c 
A2c 


T 

Ac 


A 

0 

0 


since  x(0)  =  £«  Letting 

y(o)1 


-7  = 


7(0) 

7(0) 


we  have,  from  (3.8), 
T 


2' 


0 

0 


0 

0 

0 


x(0) 

x(0) 

_ 

• 

« 

• 

x(0) 

_  .J 

X-YI  I  =<  ^7  [x(t)  -  y(t)]«  [x(t)  -  y(tj 

t=l  J 

=  |  |  [l  -  K(K'K  +  E)"3*']  J(Cx  -  Cy)||, 


1/2 


(3.18) 


* 


(3.19) 


(3.20) 


where  X  and  Y  are  both  trajectories  determined  by  (3*8)  but 
with  different  initial  conditions.  From  (3.20),  and  taking 
note  of  (3.10)  and  (3.12), 

|U( 


c(t)  -£(t)(|  =|[x(t)  -  l(t)]'fx(t)  -l(t)J  j1/2 


&  ||r-T||  S  I |l  -  K(K'K+E)-1K' 


(3.21) 


0  -G 

-x  -y 


hi 


Now,  it  can  be  shown  that  |  |  I-K(K'  K+E)-1K'  |  |  <•  1  (for  an 
indication  of  how  this  is  done,  see  the  proof  of  Theorem 
£.3«1  in  the  fifth  chapter)  and,  from  (3.11)  and  (3*18),  it 
is  clear  that  I  I jl I  =1  I a| I .  Also 


T^lf*(0)-£(°)7  'f* (0)-gr(0| 
j=l 


1/2 


(3.22) 


=  T1/2  |!x(0)-y(0)|| 


In  (3.22),  we  have  utilized  another  property  of  the  norm, 
namely  that  if  a  is  a  scalar, 

IMI  =  UMI*II«  (3.23) 

As  a  result  of  the  statements  immediately  above,  it  is  ap¬ 
parent  from  (3*21)  that 

||x(t)-i(t)||  <  ||A||Tl/2||x(0)-I(0)|h 

l/2 

thus,  given  an  e  >  0,  letting  6  =  e/(  | | A | | T  '  ),  it  is 

easily  seen  that  if  ||x(0)-y(0)[|  <  6  then  | |x(t)-y(t)| |  <-e, 
so  that  the  process  described  by  (3*8)  is  stable.  This  de¬ 
finition  of  stability  is  a  specialization  of  "Liapunov 
stability"  (see  Struble^2^). 


3«U  Sensitivity  of  the  Loss  Function  to  the  Variation  of 
Certain  Parameters 

It  is  interesting  to  examine  the  value  of 
g  min,  as  given  by  (3*7),  for  different  values  of  the  elements 
of  the  positive  definite  diagonal  matrix  E,  which  are  the 


elements  of  D  (in  (3°2))  repeated  T  times.*  This  can  be 

rigorously  investigated  by  proving 

Theorem  3«ls  An  increase  (decrease)  in  at  least 
one  element  of  E  in  (3*7)  results  in  a  decrease 
(increase)  in  (M)  as  given  therej  these 

manipulations  are  assumed  to  be  constrained  so  as 
to  maintain  the  positive-definiteness  of  E. 

Proof ;  Recall  that  K'K+Ej  and  hence  (K?K+E)-1  and 

K(K'K+E)  1Kf  j  are  positive  definite.  Now 

[K(K'K+E)“V] -1  =  I  +  (If1) 'Elf1.  (3.2U) 


Consider  the  quadratic  form 

Q  =  q'  [k( K r  K+E ) -1K -1q  =  q'q  +  p'Ep, 

where  p  =  Then 

n  p.,  P 

Q  =  S  [q.  +e  P  ], 

i=l 


(3.25) 


since  E  is  a  diagonal  matrix.  Now,  it  is  clear  from  (3.25) 
than  an  increase  (decrease)  in  any  e^_  results  in  an  increase 
(decrease)  in  Q,  for  constant  p  and  q.  let  T  denote  the 


The  intuitive  notion  that  g  min  increases  (de¬ 
creases)  if  any  of  the  control  costs  increase  (decrease)  can 

be  seen  by  regarding  the  matrix  r-K(K’K+E)  1K*  in  (3*7)  as  a 

2/2 

generalization  of  the  positive  number  i-lc  / ( k  +e ) ,  with  e  >  0. 
The  right  hand  side  of  (3*7)  can  then  be  thought  of  as  a 

generalization  of  (u-c-kz)^  [l-K^/(k^+e)]]  where  u,  c,  k,  and  z 
are  scalars.  Increasing  e  increases  the  expression  in  []  , 

2  p  2  2  *7  2 

which  in  turn  increases  (u-c-kz)  [l-1c/(k  +e)J  if  (u-c-kz) 
is  held  constant. 


■if  —1 

orthogonal  transformation  which  reduces  K(K'K+E)  K'  to 
canonical  form,  and  let 
r  =  Tq 


Then 


n 

Q=  £ 

i=l 


so  that  an  increase  (decrease)  in  Q  must  result  in  a  decrease 
(increase)  in  at  least  one  of  the  characteristic  roots 

tig,  Pn  of  [k(K'K+E)”VI  •  If  we  let  s  —  T(K2*K3—Tj)  then 

(3«7)  becomes 

n 

gmin(M)  =  (KZ-4C-U)»(KZtO-U)  -  £  t^s.2  , 

so  that  a  decrease (increase)  in  any  of  the  p,^»s  clearly 
causes  an  increase  (decrease)  in  gmin  (M,X),  Q+E.D. 

3«5  Dynamic  Programming  Solution 

A  method  of  obtaining  the  minimum  of  (3.2)  subject 
to  (3.1)  by  dynamic  programming  is  presented  in  Appendix  C. 

This  should  be  a  preferable  method  of  solution  when  the 
number  of  transitions  of  the  process,  T,  becomes  very  large. 

The  dynamic  programming  technique  substitutes  solving  T 
systems  of  equations,  each  s  x  s,  for  solving  one  sT  x  sT 
system  of  equations  as  was  done  above,  where  s  is  the 
number  of  elements  in  x(t)  and  m(t). 


*T  exists  since  K(K'K+E) 


Is  positive-definite. 


CHAPTER  IV 

SOME  ASPECTS  OF  A  1L2G  CONTROL  PROBLEM 
U  •  1  Introduction 

In  this  chapter  we  investigate  the  decomposition 
of  the  control  problem  of  Chapter  ITT  into  two  similar  smaller 
problems.  Two  goal-seeking  elements  and  each  be 

concerned  with  one  of  these  smaller  problems.  All  the  rota¬ 
tional  designations  of  Chapter  III  are  carried  over  to  this 
chapter.  In  addition,  it  is  understood  in  what  follows  that 
i  =  1,  2 ,  j  =  1,  2 ,  and  i  ^  j . 

Consider  the  following  partitioning  of  the  vectors 
x(t),  z(t),  m(t),  u(t),  and  the  matrices  A  and  Ej 
x(t)  =  [x2(t)  x2(t)]>, 

2>(t)  =  [m^t)  m2(t)]», 

£(*)  =  [%(t) 


An 

A 12 

En  0  ^ 

A  = 

E  = 

1 — 1 

CM 

A22 

0  D!2^ 

such  that  (3«i)  can  be  expressed  as 


M; 

The  two-digit  subscripts  *11*  and  *12*  are  used 
in  agreement  with  the  notation  employed  in  other  parts  of  this 
thesis « 


bh 


U5 

x^t+1)  =  A11x1(t)  +  A-j^x^t)  +  ^(t+1)  +  m^t+1)  (U.l) 

^.(t+l)  =  A21x1(t)  +  A223Cg.Ct)  +  z2(t+l)  +  m2.(t+l)  (U.2) 

and  the  loss  function  (3*2)  is  "separable",  i.e, 

g(MjX)  =  g11(M1,X1)  +  g12(M2,X2)  (U.3) 

where 

T 

gli(Mi,Xi)  =  ^  [^i(t)-u-i(t )]  tQci(-fc)-u.iCt  )3  (U.U) 

t — 1 

T 

+  ^  D^ra.Ct)  . 

The  control  nroblem  of  concern  to  G. .  is  "choose  M-  so  as  to 

li  i 

minimize  gli(Mi,Xi)  subject  to  (]j..i).« 

The  difference  between  this  control  problem  and 
the  one  studied  in  Chapter  III  is  the  presence  of  the  "cross¬ 
coupling"  disturbance  A.  .x.(t).  As  a  result,  the  total  dis- 
turbance  at  each  transition,  call  it 

— i(t+1)  =  Aij-j^  +  (U.S) 

is  functionally  dependent  on  the  state  vector  sequence"  X^. 

In  Chapter  HI  and  Appendix  C,  Z,  of  which  is  the  analogue, 


This  is  strictly  a  mathematical  statement.  It 
occurs  by  way  of  X^  in  (U»5),  hence  on  and  finally  X^  by 

interchanging  i  and  j  in  (U«5)«  An  organizational  inter¬ 
pretation  would  be  that  the  actions  taken  by  G-q  affect' 

the  trajectory  X2  of  G-^1  s  state  variables  by  way  of  (U.l) 

and  (U«5)  and  vice  versa. 


U6 


in  independent  of  X  and  the  solutions  obtained  there  are 
based  on  this  fact.  Methods  of  bypassing  this  difficulty  so 
that  these  solutions  can  be  applied  are  considered  in  later 
sections  of  this  chapter. 

We  will  be  concerned  with  finding  "equilibrium* 

.M. 

policies.  These  are  denoted  by  (M^_) e  and  (Mg)^  an<^  ^ave 
the  property  that  (M-^)  is  the  solution  to  G^'s  control 
problem  when  uses  and  vice  versa.  The  manner  in 

which  the  concept  of  equilibrium  is  utilized  in  this  thesis 
has  been  outlined  in  Chapters  I  and  II.  This  chapter  is  a 
step  in  the  application  of  these  ideas  to  certain  linear 
systems . 

jj.,2  1L2g  Control  under  perfect  Information 

In  this  section  we  make  the  following 


*The  1L2G  control  problem  formulated  here  is  an 

example  of  a  "game  of  prescribed  duration. This  nomen¬ 
clature  refers  to  a  multistage  process  in  which  each  of  n 
players  (here,  n=2)  exerts  a  control  on  the  position  of  the 
process.  In  the  case  studied  here,  the  process  is  governed 
by  (3,1),  The  control  exerted  by  the  "player"  G-^  is  through 

and  G-^'s  "payoff"  is  determined  according  to  how  well 

it  keeps  the  value  of  g, . (M.  ,X. )  down.  The  equilibrium  policy 

^  1  1  (16)'  . 
is  analogous  to  an  equilibrium  point  in  game  theory,  m 

the  sense  that  (M-^)  is  optimal  for  G^  when  G^  uses  (Mgjg 
and  vice  versa. 


k7 

Assumption  U.2.1  The  1L2G  system  is  a  "perfect 

informa  tion"^'^,‘^  system,  i.e.  G-q  and.  G^  both 

have  complete  knowledge  of  the  entire  system,  of 
U  and  2,  and  of  each  other's  goals.  Note  that  the 
statement  of  G^  t s  control  problem  in  the  previous 

section  did  not  specify  that  G-,  •  have  this  informa¬ 
tion.  1 

In  what  follows,  equilibrium  uolicies  for  G-^  and  G^,  will 

be  found  using  assumption  U.2.I.  These  policies  will  provide 

a  benchmark  with  which  to  compare  policies  found  by  other 

techniques  in  later  sections . 


Recall,  from  Chapter  III,  the  "unfolded"  repre¬ 
sentation  of  the  closed  form  solution  of  the  difference  equa¬ 
tions  (Uol)  and  (U,2),  with  notational  changes  appropriate  to 
this  chapter, 


*i  = 


■ 

I  0  0  ...  0 

—  ~ 

Ii/1)  +  “a/1) 

i±(2) 

Ai±  I  0  ...  0 

w.(2)  +  m.(2) 

t 

A2,  A..  I  ...  0 

0 

li  ii 

0 

«  « 

« 

« 

25!  (T)^ 

;t-i  .t~2  t-3  ; 

A .  .  A .  .  A  .  .  . .  o  I 
11  n  ii 

« 

Ii(T)  +  rru(T) 

~i 

_ 

Aii£i 

A2.c, 
11— i 


T 

a:.c. 

li-a. 


Ki(2i+Mi)  +  ci. 


(U.6) 


1*8 


Equation  (U«?)  can  be  represented  as 

0  0  ...  0  0 
0  0 
0  0 


*i  = 


^C1) 

S±(2) 

= 

r - 

£ 

1 . 

L 

r 

Aij  ° 


Aij  "• 


A.  .  0 


ij(!) 

i3(2) 

X.P) 

»■  « 

^(!) 

5l(2) 

*4  (T ) 


Lijlj  +  5i 


The  loss  function  (U«U)  is  rewritten  as 

gu(Hi,Ji)  =  di-Hi)’ Cli-Hi)  +  MitEiiMi 

where 


^li  ®  •••  0 


Eli  = 


'li 


'lU 


(li«7) 


CU.8) 


in  accordance  with  the  unfolded  representation. 


From  (3*6)  in  Chapter  in,  it  is  seen  that  the 
minimum  of  g^i^i’Hi)  st‘bjec't  to  (1**6)  occurs  for 

*L  »  V  (Hi-ItjJi-cp  (k.9) 

If  we  combine  the  system  of  equations  (U«9)  for  i=l  with  the 
system  (U.9)  for  1=2,  we  get  a  set  of  sT  linear  algebraic 
equations  in  the  sT  unknown  elements  of  and  M^.  In  order 
to  express  these  in  terms  of  C,  Z,  and  U  only,  it  is  necessary 


to  eliminate  the  ^  and  ^  which  is  implicitly  contained  in 
^2  and  W^,  respectively.  Also,  it  will  be  helpful  to  rewrite 


the  system  of  equations  (k*9)  as 


s-1 


(U.lo) 


Substituting  (iu7)  into  (U.6),  it  follows  that 


X.  =  K.L. .X.  +  K.Z.  +  K.M.  +  C.. 

■*“  •*-  J  J  X  a.  -L  "J- 


(U.11) 


Now,  (i|.6)  with  i  replaced  by  j,  substituted  into 
(U.ll)  yields 

h  “  Ki“i +  +  aj  +  *  Sj  (‘‘•“J 

which,  from  (U»7)  with  i  and  j  interchanged,  is 


h  =  KiMi  +  *±hj*fy±h  +  KiWjk 


+  hVjEj  *  hh&  +  Sj.  +  % 


CU.13) 


or 


,-1/ 


+  KiLij^j+Ki^i4^.i^  ’  (U.Uu) 

Equation  (U« lit)  is  an  expression  of  in  terms  of  and 
M. ;  hence,  using  (U.ll;)  with  i  and  j  interchanged,  (U.lo), 

J 

and  (U«7):>  the  following  result  is  obtained; 


-1 


So 


Now,  taking  i=l,  j=2,  then  i=2,  j=l  in  (U.15>), 
algebraic  manipulating  yields 

for  i=l,  3=2;  +  Q12M2  =  R-^ 

for  i=2,  j=l;  =  *2> 


a  little 


(U-16) 


where 


«±i  =  Ki  *  (Ki')']sli  *  ^V^jViV’Vii'V 

«u  = 

Si  '-Si  -  Kilij )"1(|CihilcA*'t31jiSi 

+  “  %%  -  %  (U.17) 


The  value  of  obtained  by  solving  the  linear  system  (U.15) 
is  the  optimal  policy  from  G-qTs  viewpoint  when  G-^,  uses  M^, 
and  vice  versa,  since  (ij.10)  is  satisfied. 


Because  of  assumption  U.2.1  (perfect  information), 
G-J2  and  O-j.2  can  envisaged  as  arriving  at  the  system  of 
equations  (U«l6)  independently.  Then,  solving  this  system, 

G-^  determines  its  control  policy  from  the  solution,  dis¬ 
regarding  If. .  There  need  be  no  communication  between  G-q  and 
G-^2  during  this  procedure.  We  now  investigate  the  effect  of 
relaxing  assumption  U.2.1. 


4,3  An  Iterative  Procedure  for  Determining  Optimal  Equili¬ 
brium  Strategies  without  Perfect  Inf ormation 

In  this  section  the  assumption  of  perfect  informa¬ 
tion  is  replaced  by 

Assumption  4.3.1.  G^  as  aware  °nly  that  part 

of  the  causal  subsystem  which  it  affects;  i.e.  it 
knows  (4.i)  but  not  Also,  G^  has  no  know¬ 

ledge  about  G^2,s  g°al  and  'vice  versa. 

Under  assumption  4*3«1,  G-q  and  G-^,  cannot  determine  equations 

(4.16) ,  as  they  were  portrayed  as  doing  in  the  previous  sec¬ 
tion.  However,  under  certain  conditions,  the  solution  of 

(4.16)  can  be  found  by  the  iterative  procedure*  described 
below. 


A  temporal  repetition  of  the  finite  process 
governed  by  (4«1)  and  (4«2)  is  imagined  to  occur  as  follows* 
G-n  assumes  w. (t)=0  for  t=l,  2,  ...,  T,  where  w.  (t)  is  given 

-LJ.  -J.  — «  —CL 

by  (4«£).  Using  this  particular  sequence,  which  we  denote 
by  (Wi)1,  the  operation  (M^)^  corresponding  to  can  be 

determined  from  (3«6),  reproduced  here  with  appropriate 
rotational  changes. 


*Which  can  be  thought  of  a  "fictitious  play**^) 
of  the  game  described  in  the  preceding  footnote, 

MM 

or  from  the  dynamic  programming  solution  of 

Appendix  c» 
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<«i>n  “ 

Then,  the  trajectory  (X-^)^  is  obtained  from  (U»6),  rewritten 
as 

%)„  -  %  ®1)n  *  <st1)n  •*  c±.  (u.w) 

Gjg  is  now  given  the  information  about  (X^)i»  ^rom  'which  it 
can  determine  (Tt2)2  using  (1|. 7),  that  is, 

®i>„  =  VV-l  *  ii-  ft-20) 

Next,  G-^2  obtains  (M2)2  and  (Xj^  Just  as  G^  arrived  at  its 
corresponding  sequences  above.  Then  G-^  is  given  the  informa¬ 
tion  about  (X2)2  i'rom  which  it  can  determine  and  the 

.g. 

whole  procedure  is  repeated.  Continuation  of  this  generates 
the  sequences 

^—1^1’  (—1^3’  QLl)£>  ’**  (li*21) 

(Zz)2>  (hh*  (-2)6’  •••  •  {h,22) 


-g 

The  iterative  procedure  that  is  carried  out  by 
Gfi  and  G^  could  be  classified  as  "adaptive  behavior"  or  as 

a  "learning  process."  What  each  element  G^  does  is  to  modify 

the  vector  parameters  which  are  the  elements  of  the  sequence 

W.  at  each  iteration  to  take  into  account  "new  information" 
x 

acquired  during  the  previous  iteration.  The  adaptive  behavior 
ceases  if  the  difference  between  adjacent  0T^)n's  is  less 
than  some  prescribed  number  6  . 


5>3 


We  now  prove 

Theorem  1*«3«1.  A  sufficient  condition  for  the 
convergence  of  (X^,  (Xi)2+i>  C^)^,  ...  is 

I  |  <■  1. 

CU*  23) 

Proof.  Substitution  of  (U.18)  into  (U.19)  yields 

<ii>n  =  Ki'Ii’„*!ci(Ki'Ki+Eii>'1'ci’C"i-Iti'5ib^i]  +  it 
=  K1[l-(IC1'K1*E11)":Llti,Kj(51)n  *  q±  ,  (ll.21t) 

where 

Si  =  Si+Ki(%'  Ki+Eli)~lKi'  (2i“Si)«  CU.25) 

Substitution  of  (U.20)  into  (l|,2l|),  along  with  rewriting 

M  gives 

(Jib  -  •  (ll-26) 


where 


Ei  = 


(It. 27) 


Now,  if  we  substitute  (I;, 26)  with  i  replaced  by  j  and 
n  replaced  by  n-1  into  (U.26)  as  it  stands,  we  obtain  the 
important  result 

«i>n  =  Kj*Eil"1 


(U.28) 


where 


(It. 29) 


It  will  be  convenient  in  the  sequel  to  have  the  abbreviations 

Hi  =  (1,-30) 

"i =  ^i’v^i^Xi*  cu.3D 


Then,  the  mapping  (1|.28)  from  (X^)  2  to  (X^)n  becomes 

Sl>„  =  niSiWV  'l*-32> 

Subtracting  (X^)n  ^  from  (X^)n  and  taking  the  norm  of  the 
result  yields  the  distance  from  (X^)  ^  to  (X. )fij 


llc*i>n-<*1>„_2ll  =  lliIiC(>ti)„.2-tsi)nJII 

5  Hn1IMlS1)n^-(i1)^ll.  CU.33) 

In  other  words, 

«0i1)„,fii1)n.2l  <  a1<j[CSi>n_2.(ii)nJ>  0.-3U) 

where  a^  =  | |II  | | ,  Referring  back  to  the  discussion  on 
contraction  mappings  in  Chapter  II,  we  see  from  (U.3U)  that 
<  1  is  a  sufficient  condition  for  the  iterative  procedure 
described  above  and  the  associated  sequences  (l*.2l)  and 
(U • 22 )  to  converge.  This  concludes  the  proof  of  Theorem  U,3»l« 


If  the  convergence  criteria  ||ll^||  <1  appearing 
in  the  statement  of  Theorem  I*. 3*1  are  satisfied,  the  limits  of 
the  sequences  (U«21)  and  (U«22)  can  be  found  by  setting 
(X^  =  (ij_)n_2  in  (U-32)s  thus 


*as  defined  in  Chapter  III,  equation  (3*9)* 


(fc.3$) 


(x.)e  = 

where  v^  can  be  found  from  (1;.2£),  (U.27)}  and  (1|,29)»  The 

subscript  *e*»  above  is  used  as  a  reminder  that  (X^)  is  an 

equilibrial  solution  or  "fixed  uoint"  of  the  mapping  (I4..32). 

Equilibrium  values  for  (Ej.)e  .are  found  from  (U • 7)  with 

X.  =  (X.)  ,  and  the  equilibrium  oolicy  (M, )  is  given  by 
3  j  e  ■  L  e 

(U.18)  with  »n"  replaced  by  "e*.  We  now  prove 

Theorem  U«3«2«  The  equilibrium  vectors  C%)e 
and  (JL,)e  determined  in  the  iterative  pro¬ 
cedure  described  above  satisfy  equations  (U.16). 

Proof,  The  equilibrium  vectors  (M^)g,  (Wj_)e >  (£j_)e  are 

governed  by  the  same  functional  relationships  as  their 

counterparts  ,  Jt,  and^^  in  section  U.2.  Hence,  an 

•g. 

identical  argument,  except  for  the  n()e"  around  each  se¬ 


quence,  can  be  used  to  arrive  at  a  system  of  equations 
identical  to  (U.l6)  with  (M^)  and  (M,,)e  as  un^n0Tn 


*•* 


In  summary,  we  have  developed  an  iterative 
procedure  to  obtain  the  equilibrium  policies  without  making 


This  argument  must  be  carried  out  from  the  re¬ 
searcher'  s  viewpoint,  since  under  assumption  U«3»l  G-^  does 

not  have  the  information  necessary  to  derive  (i|«l6). 

This  can  also  be  shown  by  direct  substitution 
of  (M^)e  into  (U«l6),  but  this  is  extremely  cumbersome 

algebraically. 
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the  "perfect  information"  assumption.  The  goal-seeking 
elements  G^  and  G  alternated  in  determining  and  applying 
their  control  actions.  In  true  simultaneous  control,  Q 
and  G-j-2  would  be  required  to  apply  the  control  actions  at  the 
same  time.  The  next  section  is  devoted  to  the  study  of  true 
simultaneous  control, 

lj.U  Simultaneous  First-Level  Control  -with  Incomplete 
Information 

The  purpose  of  this  section  is  to  investigate 
conditions  under  which  an  iterative  procedure  similar  to  the 
one  derived  in  the  previous  section  is  useful  in  determining 
equilibrium  policies  under  the  "simultaneous  control"  require¬ 
ment.  Throughout  this  section,  it  will  be  assumed  that  G-^ 
possesses  information  only  about  the  dynamics  of  the  state 
variables  under  its  cognizance,  i.e.  G-^'s  control  problem 
is  "choose  so  as  to  minimize  (i|.U)  subject  to 

x^t+1)  =  A^x.  (t)  +  w\(t+l)  +  rrn(t+l)."  (U.36) 

Although  G-j^  can  measure  the  disturbance  y^(t),  it  does  not 
know  the  mechanism  producing  n^(t),  i.e.  equation  (U*5)« 

*An  organizational  interpretation  is^22^  "When 
tasks  have  been  allocated  to  an  organizational  unit  in  terms 
of  a  subgoal,  other  goals  and  other  aspects  of  the  goals  of 
the  larger  organization  tend  to  be  ignored  in  the  decisions 
of  the  subunit," 


5>7 


The  process  to  be  controlled  assumed  to  be  an 
infinite-stage  process  which  is  periodic,  with  period  T+l, 

The  dynamics  are  governed  by  (l*.l)  and  (U.2)  with 

z,(t)  =  z.  (t+nT+n),  n  =  0,  1,  2,  ...  .  (U.37) 

—1  —X 

Also,  in  (U.l*), 

u.(t)  =  u.  (t+nT+n).  (U .  38 ) 

The  initial  conditions  are  imnosed  at  T+l,  2T+2,  ...,  so  that, 
x(nT+n)  =  c,  n  =  0,  1,  2,  ...  .  (U • 39) 

Under  this  assumption,  the  finite  T-stage  process  and  the 
associated  control  problems  formulated  earlier  are  simply 
repeated  an  indefinite  number  of  times. 

Control  of  this  process  is  assumed  to  proceed  as 
follows;  G-^  assumes  w^(t)  =  0  for  t=l,  2,  ...,  T  and  obtains 
(IL)-j*  from  (U.9)  with  =  0.  It  simultaneously  implements 
(^r±)l  observes 

5j,(t+l)  =  xi(t+l)-Aiixi(t)-mi(t+l)  CU«U0) 

as  the  process  evolves  over  t=0,  1,  ...,  T-l,  according  to 
(U.19)  with  n=l.  At  t=T,  G-^  will  compiled  the  sequence 
(Wi)i  of  "retrospective  residuals".  For  n=l,  2,  3>  . ..,  we 
use 


As  before,  subscriots  outside  the  parenthesis 
indicate  the  iteration  or,  in  this  case,  the  interval,  over 
which  the  sequence  is  used. 
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Assumption  U«U«1«  The  anticipated*  sequence  of 
disturbances  over  the  time  interval  nT+n,  nT+n+1, 
. (n+l)T+n  is  (M^)  ,  the  sequence  of  distur¬ 
bances  observed  during  the  previous  T-stage 
interval. 


The  control  action  to  be  applied  during  the  n—  period  is 
therefore  computed  from  the  formula 

<Si>„  =  ®i'%«u>'V[£r%®ib-i-5]’  (U'ki) 

n=2 j  3,  li ,  « « « , 

and  the  state  trajectory  (X^)n  is  governed  by  (U . 19 ) • 


The  strict  requirement  of  simultaneous  control 
specifies  that 

is  the  appropriate  form  of  (U.7)  to  be  used  in  this  section, 
as  opposed  to  the  * lagged"  version  (U.20)  of  the  last  section. 


If  the  sequence  (Wi)2>  •  CHi)ri>  •••  con_ 

verges,  the  anticipatory  mechanism  of  assumption  U.U.l  is 
accurate  to  any  prescribed  degree  for  n  larger  than  some 
N  which  depends  on  the  accuracy  required.  We  now  investigate 
the  convergence  of  this  sequence. 


*The  reader  may  prefer  to  think  of  as  a 
stochastic  vector.  The  prediction  is  then  indicated  by 

®ib-i . 


n— 1)  2j 


6  Q  Q 
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The  discrepancy  between  the  actual  and  anticipated 


disturbances  over  the  n—  period  is 
=  (%)„-«!  >„-i  • 

Substitute  this  into  (i;,l^)  to  get 


(U.U3) 


(*i>»  =  %[@i)n-X^!i)n+(«i)n ]  *  Si  0.  JA) 

and  substitute  for  (M^)n  in  (U.Ui)  its  equivalent  vector  as 
given  by  (U.Ul)j  then, 

«i>n  =  Ki^ih*Ki[IHVV%>"VKiJ<^h-l*%  • 

(U.U5) 

where  is  given  by  (lu25>).  The  argument  continues  along 
the  lines  of  the  argument  from  (I4. 0 2U )  to  (U.29)  in  the  pre¬ 
vious  section,  so  these  steps  are  omitted.  The  important 
result  is  the  functional  form  of  the  mapping  from  (X^)n_2 
to  <!,)„! 

(U.U6) 

From  (U.U6)  and  the  notions  of  norms  and  distances  discussed 
earlier, 

;  llnilMI (ih-a-Sih-il  1*1  lEi&5Ii)„-P!i)„.2 

*VA/j[!At3)n-1-«',3)n-3]ll 

=  IUri||<i[(x1)n.2,(xi)n4l]  +  «n,  (U.U7) 
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where  11^  and  are  given  by  (I4..30)  and  (I4.3I),  respectively. 
The  positive  term  &n  is  small,  being  a  function  of  second 
differences.  Its  lower  bound  is  zero,  hence  in  order  for 
the  inequality  (U.U7)  to  hold  for  all  possible  values  of  6n, 
we  must  have 

5  1  |Xlii  |d[cxi)n_2,(x±)n_u].  <!l.U8) 

The  control  procedure  described  above  establishes 
the  successive  application  of  (U.U6)  which,  if  |  |  |  |  <1, 

because  of  (U «U8 ) ,  generates  the  convergent  sequences 

(X rj_)i>  (X1)3,  •••■»  QLi^n-l5  **•  (U«U9 ) 

(Xi)2»  (*4)^5  ^-i^n’  ***  *  (U.50) 

We  now  prove 

Theorem  U.U.l.  If  | | TTi I |  -c  1,  the  sequences  (U.U9) 
and  (U.30)  both  converge  to  (X^)g,  the  limit  of  the 
sequence  (U«2i)  in  the  previous  section.. 

1  2 

Proof,  Let  (X^)Q  and  (X^)g  denote  the  limits  of  (U.U?)  and 

(U.50),  respectively.  Then,  taking  note  of  (U.!|2)  and  (U.U3), 

we  see  from  (U.U6)  that  the  following  equations  are  satisfied, 
1  2 

since  CS±>e  and  (X^)  are  also  "fixed  points'*  of  the  mapping 
(U «U6)  for  n  odd  and  even,  respectively) 

(ip.1  =  d-np^iutOSj).1-^),2 
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Cli)/  =  d-n,)'1 


{  Vij  [«i  ’e2-%  V]  *VWi/Ai 

[Si^-CIA2]]  , 


from  which  we  obtain  by  subtraction 

[l«(I-IIi)'ViIl3K31ji]tSi)e:L'aih2] 

=  2(i-ni)-1KA3[a))e1-QLj)92]  • 


(UcU8) 


The  two  vector  equations  generated  by  (l|.oli8)  for  i-1,  j=2  and 
i=2,  j=l  can  only  hold  if 

=  2*  =  2  * 

so  that  (Uol{6)  must  reduce  to  (i|«3f>)  in  a  state  of  equilibriumo 
This  concludes  the  proof  of  Theorem  1*  «!*«!• 


We  have  shown  that  the  equilibrium  state  for  the 
simultaneous  control  case  of  this  section  is  identical  to  the 
equilibrium  state  of  the  sinroler  iterative  procedure  of  the 
previous  section «  It  follows  that  the  equilibrium  policies 
and  (Jlg)e  are  the  same,  and  that  (M^)g  and  {Mg)e 
satisfy  equations  (1^16). 

Uoj?  Sub-optimality  of  1L2G  Control 

In  this  chapter  we  have  been  concerned  with  the 

deriviation  of  equilibrium  policies,  denoted  by  (M^)e  and  (Mg)e« 

Let  (M)  and  (X)  denote  the  over-all  policy  obtained  by  com- 
s  e 

bining  {M^)c  and  (Mg)  and  the  resulting  line  of  behavior, 
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respectively,  so  that,  from  (U«3)> 

g[(«)e.(i)J  'Snfe-IVJ  *  ei2[CM2>e*rt2>e]- 

Notice,  however,  that  the  solution  of  (U.16)  is  not  necessari¬ 
ly  the  solution  of  (3.6).  Denote  the  latter  by  (M)Qj  then 

g[(M)0,(X)o]  5  g[(M>e^X)e]  (U,U9) 

since,  by  definition  (M)q  is  the  point  at  which  g(M,X) 
achieves  its  minimum  subject  to  (3.1) «  In  the  next  chapter, 
we  introduce  a  "second-level"  unit  to  "coordinate**  the 
"action-counteraction"  procedures  of  G-q  and  G-^j  s0  that  the 
equilibrium  policy  (M)g  is  also  optimal. 


CHAPTER  V 


SECOND  LEVEL  CONTROL  IN  A  2L3G  SYSTEM 


Introduction 

The  purpose  of  this  chapter  is  to  investigate 

the  role  of  the  second  level  goal-seeking  element  in  a  2L3G 

controller  (see  Fig.  2.1).  We  will  continue  to  use  the 

control  problem  of  Chanters  III  and  IV.  Having  the  results 

of  these  chapters  available  allows  us  to  concentrate  on  the 

concepts  relevant  to  second-level  controlo  Slight  changes 

in  notation  will  be  pointed  outj  that  which  is  carried  over 

will  not  be  elaborated  upon.  When  i  and  j  are  used  as 

subscripts,  i=l,  2,  j=l,  2,  and  i^j,  as  before. 

/ 

We  list  the  assumptions  which  will  hold  in  the 


sequelj 

Assumption  ,$.1.1.  The  partitioning  of  x(t), 
mjft),  and  z_(t)  which  allows  (3«1)  to  be  re¬ 
written  as  (U«l)  and  (U.2)  is  in  effect. 

Assumption  5>«  1.2.  The  goal  of  the  second 
level  element  Gg  is  considered  to  be  the 

over-all  controller  goal.  This  is  to  mini¬ 
mize  (3.2)  subject  to  (3.1) . 

Assumption  £«I«3«  The  second  level  element 
G£  alone  has  knowledge  about  the  entire  system,. 

It  knows  the  method  of  partitioning  mentioned 
above,  the  dynamics  of  the  entire  causal  sub¬ 
system,  and  the  goals  of  the  first  level  goal¬ 
seeking  elements. 
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6k 


Assumption  The  first-level  element  G-^ 

has  the  same  information  and  control  problem  as 
in  Section  U.U«  The  process  being  controlled 
is  the  infinite-stage  periodic*  one  described 
in  that  section,  and  the  control  procedure  em¬ 
ployed  by  G-^  and  G-^  is  identical  to  the  one 

discussed  there.  G-,.'s  loss  function  is  re¬ 
written  as**  1 

T 

gli(Mi>Xi)  =  YZ  -i^'-i*^)  ' 
t-1 

+  C  v(t)Du  (s-1) 

t=i 

where  is  positive-definite  and  diagonal. 

£.2  Synthesis  of  the  Optimal  Control  Rule 

In  this  section,  G2's  control  problem  is  "choose 
U-j*  and  U2*  in  such  a  manner  that  G^  and  G^2  'will  together 
determine  M  so  as  to  minimize  g(M,X)  subject  to  (3»1)«"  We 
will  make  the 


recall  the  remarks  made  in  Section  U.U;  the 
finite-stage  control  problems  just  repeat  in  time.  An 
"optimal"  policy  will  be  understood  to  be  over  a  single 
T-stage  period  unless  otherwise  noted. 

**The  "separability"  of  g(M,X)  as  exemplified  by 
(U«3)  in  the  previous  chapter  does  not,  in  general,  hold 
here.  In  particular,  usually  we  will  have 
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Assumption  5«2.,1»  The  convergence  condition 
|  |  TljJ  |  <1  in  (U.U8)  is  satisfied]  thus,  the 

control  procedure  engaged  in  by  G-q  and  G-^> 

tends  tovrard  a  state  of  equilibrium. 

We  can  apply  the  results  of  Chapter  III  to  get 

the  sequence  of  controlled  input  vectors  which  is  optimal 

fromGg's  viewpoint.  This  is  (3*6),  reproduced  here  for 

convenience, 

(M)q  =  (K'K+E)mlKf  (U-C-l®).  (5.2) 

The  equilibrium  nolicies  (M-^)e  and  (l^g  are  found  in  Chapter 
IV  by  replacing  Wn”  by  "e**  in  (U .  18 )  5 

(Si)e  =  (q-  >  (5-3) 


where 


0  000  0 


D  *  ...  0 

lx 


0  a  0 .  D,  . ^ 

ll 


(5.U) 


in  accordance  with  earlier  usage.  Let 

(Mi)o  =[(K'K+E)"\'(U-C-I®)Ji  (5.5) 

denote  the  vector  formed  from  the  elements  of  (M)0  correspond¬ 
ing  to  the  controlled  inputs  under  G-^'s  cognizance.  In 
order  for  the  equilibrium  policies  to  agree  with  the  optimal 
over-all  policy,  we  must  have 

tti>o  =  (“it  • 

Setting  the  right  hand  sides  of  (5«3)  and  (5«5)  equal  to 
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each  other,  one  gets,  after  a  little  algebra, 

U.±*  =  Ci^Ki(W.)e+(Ki*  )"1(Ki'  Ki+Elj^)  [(F  K+E)_1K> 

(U-C-KZ)]i  .  (5.7) 

Equation  (5*6)  specifies  a  rule*  for  G2  "to  follow  in  its 
act  of  determining- the  "ideal  trajectories"  U-^  and  Up  for 
the  lower  level  units.  With  these  "distorted"  ideal  trajec¬ 
tories,  the  equilibrium  policies  arrived  at  by  G-^  and  G-jp 
using  the  control  procedure  of  Section  U.U  together  determine 
the  ootimal  oolicy. 

Notice  that  G2 ' s  rule  for  determining  Ui  ,  namely 

(5«7),  depends  on  the  equilibrium  value  (W^)e*  Because  of 

assumption-^. 1.3 3  this  is  available  to  Ggj  the  procedure  for 

finding  it  goes  as  follows: 

1.  The  trajectory  (X)Q  under  the  optimal  control 
law  is  determined  from  (3,3)  with  M  =  (M.)0* 


*It  is  interesting  to  note  the  geometrical  effect 
of  G2's  specification  of  TU*  according  to  (5«7)«  Looking 

back  to  (lj.,16)  and  (U.17),  it  is  easily  seen  that  Gg  is  adjust¬ 
ing  the  positions  of  the  hyperplanes  corresponding  to  the 
individual  equations  of  (li,l6),  so  that  they  interact  at 
the  ooint  represented  by  (M2)q3  * 
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2. 


(X)  is  partitioned  into 

Since  these  are  also  the 
mined  by  (U«l)  and  (U.2) 

brium  policies,  (W.)c  is 
r  ,  _  3-  e 


(bb  and  (X2)0. 
trajectories'  deter- 
under  the  equili- 
determined  from 


Ode  -  Lij%h+S 


(5.8) 


The  entire  control  procedure  can  now  be  envisaged 
to  occur  as  follows.  Before  getting  underway,  i.e.  prior 
to  t=0,  Gg  determines  the  ideal  trajectories  for  G  and  G-^ 
from  (5*7)«  Then,  as  the  process  evolves  as  described  in 
section  U«U,  the  sequence  of  policies 

(— i)l’  i^2s  <",as  i^n5  “0,  (5*9) 

where  (Mi)n  is  applied  from  (n-l)T+(n-l)  to  nT+n,  approaches 
the  limit  (M^)q,  as  given  by  (5.5) «  The  optimal  policy  for 
the  infinite-stage  process  from  G^'s  viewpoint  is,  of  course 
(M i^o*  i^o’  ^i^o’  *  (5*10) 

Sequence  (5.9)  therefore  represents  an  approximation  to  the 
optimal  (infinite-stage)  policy. 


*Again  we  remind  the  reader  that  our  2L3G  system 
is  contrived  to  preserve  the  autonony  of  G^^  and  thus, 

we  do  not  allow  in  the  systems  under  consideration  here, 

G2  to  «tell»  Gi;l  and  G^  the  (Wi)e  determined  in  (5.8). 

Rather  G^  is  forced  to  allow  them  to  arrive  at  (W^)g  with 

no  influence  save  UV*  according  to  (5*7). 
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Let  us  examine  the  implication  of  the  results  of 
this  section  in  regard  to  section  of  the  previous  chapter. 
From  that  section,  we  can  conclude,  in  the  spirit  of  this 
chapter,  that  if  Gg  transmits  instead  of  "distorting" 

U.  according  to  (5«7),  then  the  equilibrium  control  actions 
of  the  first  level  units  will  be  suboptimal  from  G2's  view¬ 
point.  Organizationally,  this  distortion  represents  a  co- 
ordinative  action. 

5*3  The  Effects  of  Varying 

In  this  section  we  examine  some  aspects  of  allow¬ 
ing  G2  to  adjust  the  elements  of  the  matrix  E^j*  in 
gliCMfjXi),  as  well  as  the  ideal  trajectories  U/*. 

The  abbreviations, 

”i  =  ,  (S.12) 

which  are  analogous  to  (U.30)  and  (1*.31)>  are  also  made  here. 

We  begin  by  proving  two  theorems: 

Theorem  £,3«1.  The  characteristic  roots  of  the 
positive  definite  matrix  (K^'K^+E-y*)  all 

lie  in  the  interval  (0,1). 

Proof,  Ki»Ki4-E1.*  is  positive-definite,  as  we  noted  in 
Chapter  HIj  hence,  its  inverse  and,  as  a  result,  the  pro¬ 
duct  (Ki,Ki+E-Li*)-'1E;ii#=iTi  is  positive-definite.  Therefore, 
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the  characteristic  roots  of  rr^  a.re  all  positive.  We  list 

(7) 

two  theorems  from  Bellman'  '  as  lemmas; 

Lemma.  3«3«1»  Let  A  and  B  be  symmetric  matrices, 
with  B  positive  definite.  Then 

P-k(A)  ■<  pk(A4B),  k=l,  2,  ...,  N. 

We  remind  the  reader  that  pk(A)  denotes  the  kS.  characteristic 
root  of  A,  and  recall  the  convention  stated  earlier; 

M-jj  <  5  •••  |i2  5  ^1 

Lemma  £«3«2.  If  a  symmetric  matrix  A  is  non¬ 
singular,  the  characteristic  roots  of  A  are 
the  reciprocals  of  the  characteristic  roots  of 
A.  In  particular,  if  A  is  positive  definite, 

P1(A)  =  l/p.N(A  1)  . 

Armed  with  these  two  lemmas,  and  noting  that 

"i'1  =  Wu'rX'Ki  ’  (S.13 ) 

we  see  that  the  matrix  on  the  left-hand  side  of  (5.13)  has 
characteristic  roots  all  greater  than  unity,  hence  the 
characteristic  roots  of  are  all  less  than  unity.  This 
concludes  the  nroof  of  Theorem  3»3ol* 


Theorem  3»3°2.  Simultaneously  increasing  (de¬ 
creasing)  the  elements  of  the  positive  definite 
diagonal  matrix  increases  (decreases)  1 1  |  • 

A  decrease  is  constrained  so  as  to  maintain  the 
positive  definiteness  of  E-jj*» 

Proof.  Denote  the  change  in  by  which  is  also 

positive  definite.  Then,  the  change  from  to 

E-^#  +  induces  the  change 
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Consider  the  "+"  sign  in  (5*li|)*  From  Lemma  5*3*1, 
the  characteristic  root  relationship 

*1  0ci'%)'V*+(Ki'Ki)‘: 


.  (5.XU) 

clearly 


holds,  which,  from  Lemma  5*3*2,  yields 


*»  (Eii*+  Eii*>"VKi  -=■  **N  > 


so  that 


**»  1+®ii*+  Eii*)'lKi,Ki  -  h  i+05u*r1vi'1 


Looking  back  at  (5*13),  and  again  utilizing  Lemma  5*3*2,  we 
have 


M”i>  =“1 

=  1*1  (V>'VKi+I  ^  '>*1  'V+  V> 


-1 


Ki,Ki+I 


-1 


=  fl(VW*  Eli*>  ' 

The  latter  inequality,  taken  with  the  property  that 

|  J A |  |  =p1(A)  if  A  is  positive-definite  (see  Chapter  III), 

yields 

ll(Ki'Ki+Eu'f1,!1i*  II  <■  nv1'K1+E11*+ 

<V*  %»>  II  i 

thus,  we  have  demonstrated  the  theorem  for  the  word  ^increase. " 
Wow  consider  the  sign  in  (5*1^)*  Note  that,  from  Lemma 
5,3.1  and  the  fact  that  E^*  remains  positive  definite  after 
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the  change,  that 

MEli*"AEli*)  <^k(E1.*-AEli-*HAEli*)  =Uk(Eli»)  . 

Using  this,  result,  the  proof  for  the  word  ^decrease"  is 
exactly  the  same  as  for  the  word  "increase",  except  for  the 
reversal  of  the  inequality  at  each  step  and  a  before 
h  E^-».  The  result  is 

1  |  ( K± ’  Ki+Eli^^Eli*)~1(Eli*-AEli*)  f  I 

<  [  [  (Ki'K.4Eli*)'1Eli*|  |  =  1 1  nj  |  , 

completing  the  nroof  of  Theorem  5«3*2.  Hfe  now  proceed  to 
show  how  can  use  its  capability  of  manioulating  the  elements 
of  E-y*  and  E-^*  to  assure  convergence  of  the  first-level 
control  process. 

From  (3.12),  there  exists  a  mathematical  princi¬ 
ple  by  which  we  can  bound  the  contraction  factor  |  jll^l  |  in 
(ll.l|8)  from  above* 


(5.15) 

By  Theorem  5«3«1  the  factors  |  |rr-  [  I  and  I  |ttJ  |  in  the  right 
hand  side  of  (5«l5)  are  both  less  than  unity.  Moreover,  by 
Theorem  5. 3.2,  Gg  can  adjust  E^*  and  E-^*  so  as  to  make 
these  factors  small  enough  to  assure  ||lI^||<L,  i.e,  to  assure 
that  the  sequence  (5«9)  converges.  This  is  important,  for 
without  convergence  at  the  first  level,  there  is  no  hope 
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of  Gj  effecting  any  sort  of  near  optimal  control  with  (!?•?)• 
Summary 

In  the  two  previous  sections  we  have  examined 
the  relative  effects  of  G^  varying  the  parameters  It*  and 
D-jj*  in  G-jj^'s  loss  function  (5,G.).  In  general,  we  have  seen 
that  D,^*  affects  the  rate  of  convergence  of  the  sequences 
(£,9)>  (U.U9),  (U«!?0),  and  others,  while  lb*  determines  the 
limits  of  these  sequences.  Both  of  these  variations  deter¬ 
mine  how  well  the  sequence  (5*9)  approximates  the  control 
law.  Notice  that,  whatever  the  choice*  of  E-jj*,  determina¬ 
tion  of  u  *  according  to  (£,7)  assures  that  the  limit  of  the 
sequence  (5>«9)  is  (lb)o» 

We  point  out  the  following  rather  obvious  point: 

Suppose  G2's  control  problem  is  to  determine  lb*  and  select 

D-,.*  from  the  set  d1!,  where  d1  lies  in  the  interval  fd  , 
li  '0 

d^.1),  d^xD,  in  such  a  way  to  cause  the  sequence  (5>«9)  to 
approximate  the  optimal  control  sequence  (5>.10)  as  well  as 
possible.  Then,  d^I  is  the  proper  choice  for  D-^*  and 
U.J*  is  then  determined  from  (5*7)  with 

*  .  „  _  * 
or,  if  E1± 


is  held  constant 
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This  assures  the  most  rapid  convergence  of  (5*9)  to  its  limit 
(because  of  the  method  of  choosing  If*)*  the  vector  (M.  )0* 
Appendix  D  consists  of  a  description  of  a  numerical  example 
of  how  a  variation  in  D^*  of  this  type  effects  |  |u.|  |  , 


CHAPTER  VI 


SELF-ORGANIZATIONAL  ASPECTS  OF  2L3G  SYSTEMS 


6.1  Ordering  Relations  on  Structure  Space 

In  this  chapter,  we  develop  an  ordering  rela¬ 
tion  over  a  subset  of  the  structures  of  the  2L3G  controller 
which  was  studied  in  the  previous  chapter.  The  ordering 
relation  will  be  linked  with  the  following  control  problem, 
stated  earlier  in  Chapter  IT;  "Given  a  dynamical  system 


=(?a  «>n] 


(6.1) 


where  the  ransformation  depends  on  the  structure  and  has 

a  contraction  factor  a,  find  the  structure  such  that,  given 

an  initial  value  (X)^,  the  system  reaches  equilibrium  in  the 

shortest  possible  time.”  The  contraction  factor  a  of  Q  , 

when  associated  with  the  element  of  the  structure  set  which 

induces  Q  ,  yields  a  numerical  measure  of  that  structure, 
a 

According  to  the  control  problem  stated  above  and  the  earlier 
discussion  (Chapter  II)  on  contraction  mappings,  the  smaller 
this  a,  the  greater  the  effectiveness  of  the  controller 
structure.  This  relationship  between  the  value  of  the  con¬ 
traction  factor  and  the  effectiveness  of  the  structure  allows 
us  to  rank  each  element  of  the  structure  set  according  to  the 
associated  numerical  value  of  a. 


7U 
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The  criterion  for  choosing  an  element  of  the 
structure  set  will  be  based  on  the  rate  of  convergence  of  the 
first-level  control  procedures  of  sections  U.3  and  U.iu  In 
section  5*3>  we  showed  the  dependence  of  this  convergence 
rate  on  the  matrix  parameter  of  G-^'s  loss  function  (5.1) j 

thus,  we  impose 

Assumption  6.1.1.  The  matrix  parameters  D-,-,* 
and  D-j p  ^he  first  level  units*  loss  func- 
tions  will  be  held  constant. 

This  assumption  will  allow  us  to  study  the  effects  of  struc¬ 
tural  changes  alone  on  the  rate  of  convergence. 

As  in  Chapters  IV  and  V,  i=l,  2,  j=l,  2,  and 
i-^j,  Also  in  addition  to  the  above  assumption,  Assumptions 
5.1.1  through  5.1.U  hold  throughout  this  chapter. 

6.2  Controller  Structure  and  Structural  Change 

Let  us  recall  the  statements  of  the  control 

problems  of  the  2L3G  system  of  the  previous  chapter; 

1.  G-jj;  ;  “Determine  so  as  to  minimize  (5.1) 
subject  to  (U.36).b 


Although,  as  we  pointed  out  in  a  footnote  in 


Chapter  V,  in  general 

D  f 


'V  0 


12 


Organizationally,  G  may  attach  different  “weights"  between 
the  elements  of  nn(t)  than  does 
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ft  ft 

2.  Ggt  "Determine  the  sequences  TJ-^  and  Ug 
which  will  cause  G-^  and  G-^  to  determine 
M  so  as  to  minimize  (3*2)  subject  to  (3*1) •" 

When  the  control  problems  stated  above  are  considered  collect¬ 


ively,  they  characterize  the  organizational  structure  of  the 

controller  in  the  sense  of  section  1,1,  We  now  state:. 

Definition  6,2,1,  A  member  of  the  set  of 
structures  considered  here  is  an  identical 
allocation  of  the  elements*  of  x(t),  m(t) 
and  z(t)  to  the  control  problems’  of  GT-,  and 

G-^2  seated  above.  With  each  member  of  the 

structure  set  we  can  associate  a  "selector 
vector",  =  j '0-^,  <5gj  •••>  a  s]  '  > 

specifies  how  this  allocation  is  made  as 
follows: 

dk=i,  i=l,  2,  k=l,  2,  s  if  x^t), 

mk(t),  and  zk(t)  are  under  the  cognizance*^ 


"*Mesarovic^  ^  defines  "systems  structure"  as  the 
set  of  relationships  which,  together  with  the  system's  terms 
(i.e,  state  variables,  coefficients,  etc.),  define  the  system. 
If  we  introduce  the  set  of  relationships  ,  where  "x^ft) 

rhk  Xk^W  means  "^(t)  and  xk(^')  are  under  the  cognizance 


of  the  same/different  (whichever  is  annropriate  under  that 
particular  allocation)  first  level  goal-seeking  element,"  then 
it  is  clear  that  every  allocation  of  elements  to  G.,  and  G-, 


determines  a  unique  set  of  relationship) s 


11  +  12 
The  set  fr 


M 


is,  of  course,  only  a  subset  of  the  total  number  of  relation¬ 
ships  between  the  terms.  For  example,  in  addition  to 

there  are  the  relationships  "=", 
and  (U.2). 


N 


etc.,  included  in  (U.l) 


**. 


"A  slight  conceptual  difficulty  arises  from  the 
fact  that  in  (U«36),  z^(t+l)  was  lumped,  along  with  A^jX^  (t), 


into  wi(t+l).  However,  the  control  process  of  section  Lt.U  is 

identical  whether  G, .  is  allowed  specific  knowledge  of  z.  (t) 
or  not.  ia 


77 


Henceforth,  we  will  refer  to  an  allocation  of 
the  elements  of  x(t)  only;  within  the  framework  of  the  control 
problem  considered  here,  this  induces  a  similar  allocation  of 
m(t)  and  z(t)  by  way  of  the  partitioning  of  (3,1)  into  (U»l) 
and  (U.2). 

Definition  6.2.2.  A  change  in  organizational 
structure  is  simply  a  change  in  the  allocation; * 
this  induces  a  transformation  on  the  selector 
vector. 

For  example,  a  "change  of  structure"  would  occur  if,  say, 
x^(t)  were  somehow  moved  from  G-^?s  cognizance  to  G-^'s;  "this 
would  cause  a  change  of  the  control  functions  of  all  the  goal¬ 
seeking  elements,  although  the  2L3G  system  configuration  remains 
the  same.  would  transmit  uk*(t)  to  instead  of  G^,  and 
G^  would  no  longer  be  concerned  with  monitoring  x^(t)  or 


This  particular  change  would  induce  the  transformation  1—^2 
on  <3^.5  the  other  elements  of  £  remaining  unchanged. 


*Which  induces  a  change  in  the  set  of  relation¬ 
ships  [r  J  .  Structural  changes  which  would  alter  the. 

mathematical  relationships,  such  as  replacing  "="  by  ®-^*, 
are  not  considered.  Neither  are  changes  in  configuration 
where,  say,  one  of  the  first  level  goal-seeking  elements  are 
removed. 
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Consider  a  change  of  the  type 


(V1*  V2^ ^V2*  v1); 


(6.2) 


i.e.  "change  x^(t)  fromjx^(t)  to  x^(t),  and  change  x^(t)  from 
Xp(^)  i°  — This  also  causes  similar  changes  between 
the  control  input  vectors  rn^(t)  and  mg(t),  the  disturbance 
vectors  jz^(t)  and  z2(t),  and  tiie  ideal  trajectories  u*(t)  and 
u*(t).  This  change  induces  a  row-column  interchange  in  the 
matrix  A  as  well: 


11 


1-  .  «  «  ,  d_ 

lk  ,  lp 

I 


3,  n  «  «  •  3,  . 

kl  kk 


kp 


Is! 


ks 


3  .  •  '  •  3  •  «  «  3  I 

pi  pk  |  pp  ps 


3  n  *  •  •  3  .  *  «  «  3  «  «  «  3 

si  sk  sp  S3 


^all^ll* *  ’  ^alp^lk**  ’  ^alk^lp*  * '  'als^ls, 


(3  n  C  f  3  .  •  •  •  f  el  _  ) 

v  pl'kl  '  pp'kk  i  v  pk  kp  x  ps' 

»  *  ,  *  • 


ks 


^akl^pl* ' ‘ ^akp^pk* 

%  * 

«  « 

(asl>sl*"(aspW 


*  ^akk^pp*  *  *^aks^ps 

«  • 

•  « 

'  ^ask^sp*  *  *  ^asl^sl 


The  heavy  dashed  lines  indicate  the  partitioning  of  A  induced 
by  the  controller  structure.  The  subscripts  inside  (outside) 
the  parentheses  indicate  position  in  the  matrix  before  (after) 
the  change.  There  is,  therefore,  an  interchange  of  elements 
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between  the  matrices  A^,  A^,  A  ^  and  A ^  as  a  result  of 
the  change  in  controller  structure  indicated  by  (6.2).  For 
example,  in  A^j  the  pHi  column  is  transformed  as 


- 1 

tu 

alk 

a2P 

a2k 

• 

« 

« 

• 

a 

• 

a 

a  i 

SpP 

sijk 

and  the  k —  row  suffers  the  transition 

(ak,Sl+l  ak,s1+2  “•  ak,s> - ^ap,Sl+l  ap,3l+2  **• 

where  is  the  number  of  elements  in  x^(t).  We  now  proceed 
to  indicate  how  a  structural  change  such  as  the  one  described 
above  affects  the  contraction  factor  ||ll\]|  of  the  mapping 
(U.U8)  associated  with  the  first-level  control  procedure  of 
section  U.U. 

6.3  Evaluation  of  the  Effects  of  a  Change  in  Structure 

In  this  section,  we  shall  discuss  the  dependence 

of  the  transformation  Q  in  (6.1)  on  the  structure  under  which 

a 

the  controller  is  operating. 

Note  that  ( J II ^ | |  in  (U«U8)  is  the  norm  of  a  pro¬ 
duct  of  several  matrices  (see  I;. 30),  each  of  which  is  altered 
by  a  structural  change.  Looking  back  to  the  upper  bound  of 
lllljl  established  by  (£.!!?),  we  see  that  a  reasonable 
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starting  point  of  the  analysis  of  this  section  is  to  examine 
how  changes  in  K^’K^  affect  llnji,  where  rr^  is  given  by 
(f>.12)$  thus,  we  prove  the  three  theorems* 

Theorem  6«3«1 


S  ||(Ki'Ki+Eli*)~1Eli*||  . 


Proof r  From  (3.12), 

II*! i*ll  =  1 1  (Ki'K^E^*)  (Ki'Ki+Eli*)_-1Eli*|  | 

*  (I  IVKi' i+!  lEli*l 1)11  ^Ki,Ki+Eli*)~lEli*l  i» 

from  which  Theorem  6.3.1  follows. 


Theorem  6.3.2  If  the  elements  of  E  *  are 
identical,  and  equal  to  e^/*,  then 


(Ki'Ki+Eii*)~1Eii*|| 


li 


(6-3) 


Proof:  Let  p  be  an  orthogonal  matrix  such  that 


Ki'Ki=P 


Ul 


^2 


P» 


(  j) 

Thei  according  to  Bellman'  " 

(VKi«u*)JV  =  (q'^iiV)-1.^ 


=  P 


’Si*8!!*'  eii* 


P*  • 
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The  diagonal  elements  in  the  matrix  immediately  above  are 
the  characteristic  roots  of  so  the 

theorem  follows  immediately  from  property  (3.17). 


Theorem  6.3«3  An  increase  (decrease)  in  the 
smallest  characteristic  root  of  (E-jj*)""^K^rK^ 


results  in  a  decrease  (increase)  in 

1 1  (K1'Ki«11*)-1Eu*|  I 


Proof*  Let  P  be  an  orthogonal  matrix  such  that 


As  in  the  proof  of  Theorem  6.3.2, 


i+(Eu*)’JVKi  =  p 


1+lir 


Hit 


N 


P» 


so,  an  increase  (decrease) in 


Lemma  5»3«2  and  equation  (5>.13),  a  decrease  (increase)  in 

^il^Ki'Ki+Eii*)”\i*l  =  1 1  (Ki'Ki'tEii*)""lE1i*l  I  =  IUJU 

QsE.D. 


In  the  previous  section,  the  manner  in  which  a 
structural  change  induced  a  reshuffling  of  the  elements  of 
A  between  A^,  A12>  A^,  and  ™as  examined.  Recalling 
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the  definitions  of  K.  and  L.  ,  we  see  that  a  structural 
a  ij 

change  would  alter  the  elements  of  these  matrices,  with  a 
subsequent  change  in  the  scalars  j|K^||  and  1 1 |  |  •  Also, 
from  the  theorems  proved  immediately  above,*  the  resulting 
change  in  K^'K.^  affects  the  number  1 1  rr^ 1 1  •  Therefore,  the 
factors  in  the  upper  bound  (%. 1 £)  of  ||n^||  all  change  when 
the  structure  is  altered.  Theorem  6.3*1  illustrated  how  a 
decrease  in  1 1  r K.  1 1  forces  a  lower  bound  of  ||rr.||  to 
increase.  Although  the  inequality  expressed  by  Theorem 
6.3.1  is  somewhat  weak,  it  illustrates  that  decreasing 
1 1  Ki' %!  !  s^ou^  be  avoided  if  one  were  attempting  to  decrease 

1 1  "ill* 


Theorem  6.3.2  gives  us  some  insight  as  to  how 
|  |  Tfi |  |  varies  with  changes  in  the  minimum  characteristic 
root  of  Because  of  the  relationship  expressed  by 

Lemma  5.3*1,  Theorem  6.3*2  can  be  of  value  in  studying  the 
effects  of  structural  changes  on  1 1  tt^J  | »  Theorems  6.3*2, 
5>«3»lj  and  5*3.2  enforce  the  intuitive  notion  that  positive 
definite  matrices  are  generalizations  of  positive  muifcers. 
Carrying  this  intuition  a  little  farther,  we  can  examine 


*and  the  fact  that  p,  (K^’K^)  is  equal  to  | K^T | » 

the  determinant  of  K.'K-.  hence  is  a  function  of  all  the 
1  1 

elements  in  K^’K^* 
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(U »30)  and  (£,l£)  and  conclude  that,  if  it  were  possible  to 
effect  a  structural  change  which  would  cause  only  a  "slight** 
change  in  the  produce  1 1  K]_|  |  *  1 1  Kg|  |  •  |  |l12|  f  •  |  lL2ill  while 
increasing  ||k^*K^||  considerably,  this  would  be  a  "change  for 
the  better,**  as  it  would  tend  to  decrease  |  (n^l  |  and  hence 
an  upper  bound  of  |  |u  |  | , 

Theorem  6 • 3 • 3  is  an  attempt  to  resolve  the  appar¬ 
ent  narrow  application  of  Theorem  6«3«2,  due  to  the  requirement 
that  all  the  diagonal  elements  of  E^*  are  identical.  These 
two  theorems  therefore  complement  each  other.  Theorem  6.3.2 
giving  an  insight  as  to  the  numerical  behavior  of  |  |rr.  |  |  as 

P'N^i' %)  varies,  and  Theorem  6.3*3  providing  the  generality 
to  make  this  insight  useful. 

While  it  is  possible  to  numerically  determine 
the  effects  of  the  changes  in  the  factors  in  the  upper  of 
||ll^||  expressed  by  (5>,15>),  it  is  more  fruitful  to  calculate 
| jll^l |  directly  when  the  computer  is  resorted  to,  since  this 
only  involves  the  computation  of  one  maximum  characteristic 
root  instead  of  the  six  necessary  to  determine  the  right-hand 
side  of  (5.13)* 

In  Chapter  IV,  we  found  the  mapping  between 
(X i)n  and  (X^)n+2  an^  the  conditions  under  which  this  mapping 
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is  a  contraction  mapping,  namely  J|lli||<d.  The  following 

theorem  allows  us  to  easily  determine  the  contraction  factor 

for  the  mapning  between  00n  and  OOn+2>  over-all  system 

state  trajectories  over  the  nHi  and  (n+2)—  T-stage  control 

periods,  respectively,  once  Ull-jjl  and  ||lT2|[  are  known. 

Theorem  6.3.U  The  contraction  factor  a  of  the 
mapDing  between  (X)  and  (X)n+2  induced  by  the 

iterative  procedures  between  G^  and  G^  described 

in  sections  U.3  and  k»h  is  equal  to  the  maximum 
of  the  contraction  factors  |jlT_||  and  | [ IXg 1 1  of 

the  mapnings  between  (X. )nand  (X^)n+2  for 

i=l,  2,  respectively. 

Proof*  Let 


From  (3,15), 

1 1  Hill  =[^1(ni,ni)J  1/2  • 

Let  be  an  orthogonal  matrix  such  that 
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1 

i^i(tii'tii) 

.  0 

r  n 

2- 

*1  0 

v.Tw 

0 

r-i 

ty 

0  Q2 

p1(ii2'h2) 

0  q2» 

0 

_ 

Vs  T(ii2'ii2) 

(6.U) 

where 

- 

- 

Qi  o' 

Q  = 

- 

O 

ro 

is  also  an 

orthogonal  matrix.  Looking  at  (6«U)  it  is  easy 

to  see  that 


a 


=  I  lull  =  r^cii'ii)]1/2  = 


from  which  the  theorem  follows. 


Theorem  6«3«U  is  convenient  from  a  computational 
viewpoint,  since  it  allows  us  to  use  the  smaller  matrices  11^ 
in  the  maximum  characteristic  root  algorithm  instead  of 
the  full-size  matrix  U.  Also,  the  factors  |(ll.||  are  of 
interest]  after  these  are  obtained,  because  of  Theorem.  6.3. U, 
no  additional  computing  is  required  to  obtain  the  "over-all" 
contraction  factor. 


The  control  procedure  for  the  2L3G  system  under 
study  here  is  reviewed  at  this  point:  an  iterative  procedure 
is  induced  by  the  "interplay"  between  G  and  G-^,.  If 
j  |ll,  |  |  <3L,  this  procedure  converges  in  such  a  manner  so  that 
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the  "equilibrium  control  actions"  of  G-^  and  G are  optimal 
from  G2's  viewpoint.  This  is  accomplished  through  G2's 
"influence"  via.  adjustment  of  the  oarameters  t )  in  Gii-s 
loss  function  according  to  (5.7).  Ideally,  G^'s  optimal  state 
trajectory  would  be 

(X)0.  Q0o»  (x)0»  (6*6) 

where  (X)  is  the  T-stage  sequence  of  state  variables  under 
the  optimal  control  law  (M)Q,  obtained  from  (3*6).  However, 
the  state  trajectory  under  the  collective  control  action 
exerted  by  G^  anc*  G^  turns  out  to  be 

(X)l»  (X)2»  ( Dy  (6.6) 

a  sequence  whose  limit,  due  to  G2fs  influence,  is  (x)Q.  The 
faster  (6.6)  converges,  the  better  (6.6)  approximates  (6. £), 
Theorem  6.3.1*  and  equation  (U.30)  allow  us  to  compute  the 
contraction  factor  of  (6.6)  for  a  specific  element  of  the 
structure  set,  i.e.  allocation  of  state  variables  between 
Gi;l  and  G12« 

Suppose  two  different  structures  were  being  com¬ 
pared.  Further,  assume  that  the  contraction  factor  ||ll||  is 
smaller  for  one  structure  than  for  the  other.  Then,  clearly 
we  could  classify  that  structure  with  the  smaller  ||ll||  as 
the  "better"  of  the  two,  since  (6.6)  would  converge  faster 
when  the  2L3G  system  was  operating  under  that  structural 
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configuration,  resulting  in  a  better  approximation  to  (6.£). 


The  next  section  is  concerned  with  a  specific 
example  illustrating  the  variation  of  ||ll||  as  the  structure 
is  changed. 


6.U  Self -Organizational  Aspects 


In  this  section,  we  make 

Assumption  6.U.I.  The  second  level  goal-seeking 
element  in  the  2L3G  system  under  study  here 

has  the  capability  to  change  the  structure  of  the 
controller  within  the  framework  specified  by 
Definition  6.2.2,  in  addition  to  the  capability 
of  setting  u^ft)  in  G-^'s  loss  function. 

This  assumption  and  the  remarks  made  previously  in  this  chap¬ 
ter  compose  the  formulation  of  a  "structural  choice"  control 
problem  for  similar  to  the  one  stated  at  the  beginning  of 
this  chapter.  For  every  element  of  the  structure  set,  the 
2L3G  control  procedure  described  in  the  previous  section 
holds.  No  matter  what  structure  the  2L3G  system  operates 
under,  the  sequence  (6.6)  converges  to  (X)0«  This  is  due  to 
the  capability  of  Gg  to  manipulate  u^*(t),  which  it  does  ac¬ 
cording  to  (5o 7) j  thus,  under  any  structural  configuration  the 
limit  point  of  (6,6)  is  (X)Q«  We  now  make 

Conjecture  6.U.I.  For  the  2L3G  system  studied 
here,  there  is  a  different  value  of  } |ll| | ,  the 
contraction  factor  of  the  mapping  between 
(X)n  and  (X)n+2  induced  by  the  2L3G  control 

procedure,  for  each  element  of  the  structure 
set  considered. 
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yfe  state  this  as  a  conjecture  rather  than  a  theorem  because 
of  the  immensely  cumbersome  algebra*  associated  with  the 
proof  of  the  statement.  Ife  cite  the  remarks  of  the  previous 
section  as  evidence  as  to  the  plausibility  of  conjecture 
6.4.1.  Operationally,  this  conjecture  can  be  proven  true 
or  false  for  any  special  case  of  the  system  studied  here,  by 
numerically  evaluating  ||ll||  for  every  possible  structure. 
For  example,  let 


2 

U 

1 

1 

8 


3  4  1 
2  11 

4  3  3 
16  2 
2  3  8 


(6.7) 


in  (3«1)  and  Dqj*=I  and  D  *=T  in  (5.1).  Consider  the  (g)=10 
possible  structures  with  two  state  variables  in  x^(t)  and 
three  state  variables  in  x^ft).  Table  VI-1  shows  the  results 
of  a  computation  using  the  definitions  of  and  implicit 
in  (4.6)  and  (4«7)>  (4.30),  (5*4)  and  Theorem  6.3*4.  In  this 
example,  T=3.  It  is  seen  that  only  four  of  the  ten  structures 
would  be  considered  ^feasible"  i.e.  have  ||ll||<L,  for  the 


*For  the  norm  considered  here,  one  would  have  to 
determine  p.^(TT^'II^)  as  a  function  of  the  elements  of  tt2» Ki> 

K2,1'12’  and  L21  “u^iplied  together  in  the  appropriate  order. 


I 

I 

\ 
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assumed  values  of  These  are  structures  3>  7>  9>  and 

10. 


TABLE  VI-1 


structure 


selector  vector 


null 


1 

2 
3 
h 

5 

6 

7 

8 
9 

10 


(11222)  1*879 

(12122)  1.026 

(12212)  .935 

(12221)  1.U32 

(21122)  1.251 

(21212)  1.376 

(21221)  ' .692 

(22112)  1.331 

(22121)  .8U6 

(22211)  *808 


We  will  classify  G2's  choice  of  a  structure  for  the  2L3G 
system  as  » self-organizational  activity."  One  type  of  self- 
organizational  activity  is  for  G2  to  determine  a  table  similar 
to  Table  VI-1;  then,  if  Conjecture  6.U.1  is  found  to  hold, 
select  that  structure  for  which  [|ll||  is  a  minimum. 

It  is  interesting  to  note,  from  (U.30),  that 
| j II | |  is  independent  of  the  uncontrollable  variables  U  and 
Z.  Consider  the  following  situation.-  our  2L3G  controller,, 
concerned  with  guiding  the  causal  subsystem  so  as  to  minimize 
(3.2)  over  each  time  period  subject  to  (3*1)  with  A  given  by 
(6.7),  has  carried  out  the  control  procedure  outlined  in  the 
previous  section  with  the  additional  act  by  G2  of  determining 
the  "best"  structure  from  Table  VI— 1«  As  the  process  evolves, 
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(6.6)  tends  toward  its  limit  (X)Q  at  the  fastest  rate  possible 
within  the  restrictions  imposed  by  the  structure  set  consider¬ 
ed.  The  effect  of  an  alternation*  in  U  or  %  is  to  change  the 
optimal  control  law  (see  (3*6)  ),  with  a  resulting  shift  in 
the  "influences'*  U-j*  and  U2*  (according  to  (5*7)  )  and  the 
equilibrium  policies  arrived  at  by  G^  and  G^  under  these 
influences.  However,  no  "reorganization",  i.e.  structural 
change  by  Ggj  is  necessary,  because  of  the  functional  inde¬ 
pendence  of  | | IT | |  on  U  and  2*  When  a  change  in  U  or  Z  occurs, 
we  can  imagine  t  as  being  set  equal  to  zero  with  the  control 
procedure  in  question  starting  over  from  the  beginning,  with 
the  exception  that  G2  need  not  redetermine  the  optimal 
structure.  The  rate  of  convergence  towards  equilibrium, 
therefore,  remains  the  same. 

Suppose  instead  that  the  elements  of  the  matrix 
A  are  subject  to  change,  this  change  being  immediately  de¬ 
tectable  by  G2«  Then  Gg  is  forced  to  re-examine  the  contrac¬ 
tion  factor  for  all  possible  structures.  A  specific  example 
serves  to  illustrate  this  point.  Suppose,  in  (6.7),  a^  and 


^Organizationally,  a  change  in  the  ideal  trajectory 
U  mi^it  be  termed  a  change  in  the  objectives  of  the  organiza¬ 
tion,  while  a  change  in  2  would  correspond  to  an  environmental 
alteration. 
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a ££  are  abruptly  charged  from  8  to  I*.  Table  VI-2  shows  the 
results  of  a  computation  similar  to  the  one  that  led  to 
Table  VI-1.  According  to  these  tables,  G2  should  leave  x^ft) 
■under  G-^'s  con^ro-*-  an<^  shift  each  of  the  other  state  vari¬ 
ables*  from  its  existing  structural  location  to  the  alternate 


TABLE  VI-2 


structure 


1 

2 

3 

k 

5 

6 

7 

8 
9 

10 


[ill 

l;Wi6 

.835 

.773 

1.U73 

1.235 

1.182 

*912 

1.225 

.795 

.817 


one.  After  this  structural  change  is  effected,  the  control 
procedure  evolves  exactly  as  before.  From  the  researchers 
viewpoint,  this  activity  appears  as  teleological  self-organi- 
zationj  the  structural  configuration  of  the  2L3G  controller 
is  altered  in  response  to  a  change  in  A  in  order  to  insure 
that  (6.6)  continues  to  approximate  (6,5)  as  well  as  possible. 
The  structure  choice  problem  stated  in  section  6.1  is  solved, 


*The  change  from  structure  7  to  structure  9  might 
be  more  plausible  if  a  ncost*  is  associated  with  each  altera¬ 
tion.  This  would'  involve  G-q  and  G-^  •’trading*  x^(t)  and 

x3(t)  only. 
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since  that  structure  -which,  causes  (6.6)  to  be  the  best 
approximation  to  (6.5)  is  also  that  structure  that  brings 
the  system  into  equilibrium  as  quickly  as  possible. 


One  more  example  serves  to  illustrate  these  ideas 
further.  Organizationally,  this  might  be  termed  "reorganizing 
after  a  merger."  Consider 


TABLE  VI-3 


structure _ selector  vector _  II 

1  (1122)  1;006 

2  (2121)  .690 

3  (1221)  .959 


even  number  of  state  variables,  in  this  case  four,  it  is 
necessary  to  consider  only  (l/2)(^)  possible  structures, 
since  nothing  is  gained  by  calculating,  say,  |jll||  for 
(2211).  To  continue,  suppose  x^(t)  is  added  to  the  state 
vector,  its  interactions  -with  the  other  four  state  variables 
and  itself  being  determined  by  the  fifth  row  and  column  of 
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A  in  (6.7)*  Tablp  VI-1  indicates  that  should  take  over 
x^(t)  from  G|2  and  latter  should  incorporate  x^(t)  into 


its  bailiwick. 


6.5  Self -Organizational  Activity  from  the  State-Transition^ 
Viewpoint 

In  Appendix  C  we  derive  the  dynamic  programming 
solution  of  the  control  problem  of  Chapter  m.  This  solution 
is  numerically  equivalent  to  that  given  by  (3*6),  but  the 
philosophy  of  implementation  is  somewhat  different.  Tfhile 
(3*6)  yields  the  sequence  of  control  actions  nail  at  once” 
so  that  these  actions  are  known  before  the  process  begins, 
the  dynamic  programming  solution  yields  a  formula  of  the  type 

m(t+l)  =  mj^(t),u(t),£(t+r)j  ;  (6-9) 

thus,  from  the  state -transition  viewpoint,  the  control  actions 
are  not  determined  until  the  information  on  x(t)  is  known, 
i.e.  immediately  before  the  action  is  to  be  taken.  The 
advantage  of  the  dynamic  programming  solution  was  mentioned 
in  section  3°5« 


Ife  now  prove 

Theorem  6.3.1  Let  a  mapping  from  p  to  £  be 
defined  by 

q  =  Bp+e  ,  (6.10) 

where  B  is  a  non-singular  matrix. ' 

Then  a  necessary  condition  for  (6.10)  to  be 
a  contraction  mapping  is  ||b||«1. 


Proof*  If  (6.10)  is  a  contraction  mapping,  for  two  points 

El  and  £2’ 

l|B||  llPi-Pgll  5  I  (Bp^Pjtl  =  llq^U  <  ||prP2ll  > 

hence 

IIb||  <1 

Q-E,D«  Theorem  6.6.1  can  be  thought  of  as  a  companion 
theorem  to  Theorem  U.U.l  for  linear  mappings,  since  the  two 
assert  that  | |b| |<L  is  a  necessary  and  sufficient  condition 
for  (6.10)  to  be  a  contraction  mapping. 

Suppose  that  T,  the  number  of  state  transitions 
in  each  period,  is  large  enough  so  that  Gg  cannot  apply  (3*6) 
and  (5*7)>  nor  compute  ||ll||  as  indicated  in  the  previous 
section,  all  because  of  the  large  size  of  the  matrices  in 
these  formulas.  Assume  Qj  selects  a  specific  structure. 
Appendix  E  consists  of  a  derivation  of  the  dynamic  programming 
analogue  of  (6«7)j  which  applies  to  arrive  at  U^*,  the 
sequence  Uj*(l),  ^*(2),  ...,  u^T).  This  is  transmitted 
to  o  .  Then,  the  iterative  procedure  progresses  as  before 
except  that  G-q  and  G^  use  the  dynamic  programming  solution 
of  Appendix  C  instead  of  the  numerically  equivalent  (U.Ul). 
Now,  since  Q2  was  unable  to  compute  ||ll||  for  the  particular 
structure  selected,  it  must  attempt  to  determine  it  from  the 
state  variable  sequences.  After  five  control  periods  have 
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elapsed,  G2  can  determine  |  |  |  and  1 1  (X)^-C^)^l  I  by 

(3»9)«  Because  of  Theorem  6.5*1,  assuming.  g2  is  aware  that 
the  mapping  induced  by  the  control  procedure  is  of  the  type 

(6.10) ,  G2  need  only  compare  | | (X)1-(X)3| |  with  ||(X)3-(X)^|| 
to  determine  whether  or  not  the  chosen  structure  is  ''feasible*, 
i.e.  has  an  associated  ||ll||  less  than  unity.  Moreover,  if 
G2  repeats  this  process  for  every  structure,  it  can  determine 
the  "optimal"  structure  as  the  one  for  which 

(x)3-(x)^[  | 

||(x)1-(x)3l| 

is  a  minimum. 

From  the  state-transition  viewpoint,  the  self- 
organizational  activity  of  the  2L3G  system  is  more  like  our 
intuition  tells  us  it  should  be.  G2  selects  a  structure,  the 
process  begins,  and  if  (6.11)  is  greater  than  unity,  G2  effects 
a  "re-organization"  and  the  control  procedure  starts  over  from 
scratch.  If  (6.11)  is  less  than  unity,  G2  can  be  assured 
that  (6.6)  will  converge  to  (X)Q>  although  G2  may  try  other 
structures  in  an  attempt  to  speed  up  this  convergence. 

It  is  important  enough  to  note  again  that  we  have 
assumed  that  G2  is  aware  that  a  linear  mapping  of  the  type 

(6.10)  is  induced  by  the  control  procedure  under  considera¬ 
tion.  without  this  knowledge,  no  conclusion  could  be  drawn 


(6.11) 
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from  the  numerical  value  of  (6.11). 


6.6  Applications  in  Organization  Theory 

The  general  problem  to  which  "classical”  organi- 

(22) 

zation  theory  addresses  itself  is  the  following!'  ' 

Given  a  general  purpose  for  an  organization,  we 
can  identify  the  unit  tasks  necessary  to  achieve 
that  purpose.  The  problem  is  to  grouo  these 
tasks  into  individual  jobs,  to  group  the  jobs 
into  administrative  units,  to  group  the  units 
into  larger  units,  and  finally  to  establish  the 
top  level  departments — and  to  make  these  group¬ 
ings  in  such  a  way  as  to  minimize  the  total  cost 
of  carrying  out  all  the  activities. 

In  order  to  formulate  an  abstract  model  of  an  organization, 
then,  according  to  the  above  statement,  there  must  be  some 
measure  of  the  effectiveness  of  any  organizational  design. 
This  amounts  to  an  ordering  relation  over  the  structure  set. 
For  example,  the  "assignment  problem",  so  familiar  to  opera¬ 
tions  researchers,  gives  such  a  measure.  In  this  thesis,  we 
have  proposed  another  such  measure  and  demonstrated  its 
feasibility  by  applying  it  to  a  special  case. 


Any  normative  theory  of  organizational  behavior 

must  include  some  self -organizational  capability,  since,  as 

we  mentioned  in  Chapter  I,  reorganization  is  a  common  method 

(2) 

of  attacking  organizational  inefficiency,'  The  theory  of 
multi-level,  multi-goal  systems^^5  is  ideally  suited  to 
model  this  type  of  activity.  In  a  real  organization,  the 
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"bosses"  impose  organizational  changes  on  their  subordinates. 

In  mLnG  systems  theory,  this  is  represented  as  we  have  done  in 
Assumption  6.U.1,  by  allowing  higher  level  units  the  capability 
of  re-allocating  tasks  among  the  first  level  units. 

In  conclusion,  we  must  agree  with  Mesarovic,) 
that  a  mLnG  system  is  the  closest  that  general  systems 
theory  can  come  to  offering  an  abstract  model  of  an  organiza¬ 
tion.  This  is  reinforced  if  Churchman's^  "characteristic 
(b)"  of  an  organization, 

at  any  moment  of  time  the  organization  is 
pursuing  a  set  of  goals, 


is  accepted 


CHAPTER  VII 


SUMMARY  AND  CONCLUSIONS 


7.1  Summary 

In  this  investigation  we  have  been  concerned 
with  formulating  a  particular  general  allocation  problem  and 
solving  it  for  an  important  special  case.  The  problem  con¬ 
cerns  the  allocation  of  tasks  among  several  interrelated 
goal-oriented  control  units  which  are  collectively  concerned 
with  controlling  a  causal  subsystem  so  as  to  achieve  some 
over-all  purpose.  Some  of  these  control  units,  in  order  to 
compensate  for  their  being  unaware  of  a  portion  of  the  entire 
system  and  exerting  an  effect  on  only  a  portion  of  it,  employ 
a  form  of  adaptive  behavior  in  arriving  at  their  control 
actions.  It  is  postulated,  in  the  general  case,  that  the 
time  it  takes  for  this  adaptive  process  to  reach  an  equili¬ 
brium  state,  if  at  all,  is  a  function  of  the  manner  in  which 
the  tasks  are  allocated  among  the  control  units,  i.e.  the 
"organisational  structure"  of  this  collection  of  units.  It 
is  argued  that,  if  this  is  true,  the  rate  of  adaptation  is 
a  good  measure  of  the  effectiveness  of  the  organizational 
structure,  which  yields  a  method  of  ranking  the  members  of 
the  structure  set.  One  structure  is  termed  "better"  than 
another  if  the  first  exhibits  a  higher  adaptation  rate  than 
the  second. 
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With  an  ordering  relation  over  the  set  of  struc¬ 
tures,  such  as  the  one  proposed  in  this  thesis,  it  is  a 
simple  matter  conceptually  to  select  the  "best"  structure  'from 
the  set.  If  we  allow  one  of  the  control  units  within  the 
system  the  capability  to  select  the  structure,  the  entire 
system,  when  viewed  from  the  outside,  exhibits  what  we  have 
termed  "self-organizational"  activity;  the  system  changes  its 
own  structure  in  attempting  to  increase  its  rate  of  adaptation. 

One  characteristic  of  real  organizations  is  their 
ability  to  reorganize  themselves.  This  is  accomplished  in 
the  manner  indicated  above;  a  sub-unit,  usually  termed  a 
"manager"  or  "control  group",  carries  out  the  allocation 
and  re-allocation  of  tasks.  For  this  reason,  we  have  argued 
that  mathematical  models  of  organizations  must  have  some 
self -organizational  features. 

7.2  Conclusions  and  Future  Research 

The  latter  part  of  this  thesis,  Chapters  III 
through  VI)  is  concerned  with  developing  the  ideas  summar¬ 
ized  in  the  previous  section  for  a  linear  discrete-dynamic 
system  with  a  quadratic  loss  function.  We  conclude,  there¬ 
fore,  that  these  results  are  applicable  in  this  case.  This 
part  of  the  thesis  also  demonstrates  that  it  is  possible  to 
construct  mathematical  self-organizing  systems  which  display 
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a  number  of  conceptual  similarities  to  the  intuitive  notion 
of  an  organization. 

The  model  of  organizational  behavior  developed 

in  this  thesis  might  more  properly* be  termed  a  "simulation,** 

We  have  tried  to  imitate  some  intuitive  conceptions  of 

organizational  behavior  rather  than  predict  how  a  "real- 

world"  organization  would  operate,*  In  particular,  the  idea 

* 

that  organizational  stability  depends  on  organizational 

H2) 

structure,  as  qualitatively  argued  by  Dubin,v  '  is  nicely 
represented  in  our  model. 

The  idea  of  applying  feedback  control  theory  or 
"systems  analysis"  in  the  study  of  organizations^0^  is  not 
new.  However,  most  of  the  models  involved  in  these  studies 
are  1L1G  systems,  so  that  there  is  a  lack  of  an  appropriate 
description  of  the  structure.  In  the  theory  of  mLnG  systems, 
however,  any  organizational  structure  could  be  modelled. 


*The  reader  may  object  to  our  representing  groups 
of  humans  by  "goal-seeking  elements,"  However,  at  least  one 

eminent  management  scientist^'’)  warns  that  complete  rejection 
by  organization  theorists  of  the  "mechanical  models"  of  humans 
precludes  the  application  of  many  of  the  recent  developments 
in  the  information  and  communication  sciences  to  organization 
theory. 


101 


The  simple  2L3G  system  we  have  studied  illustrates  the  idea 
of  "influence*  or  "indirect  control",  i.e.  where  the  higher 
level  unit  adjusts  parameters  in  the  performance  function 
of  units  below  it.  This  idea  has  also  been  investigated  by 
Ackoff^  from  the  viewpoint  of  decision  theory. 

Investigations  into  mLnG  systems  with  m>2  would 
be  valuable.  In  particular,  this  introduces  "middle  manage¬ 
ment"  goal-seeking  elements  which  indirectly  control  the 
causal  subsystem  while,  at  the  same  time,  being  subjected  to 
control  from  above.  Any  self-organizational  activity  by  the 
top-level  unit  would  then  create  an  entirely  new  system  from 
the  mid-level  units*  viewpoint.  The  latter  would  then  be 
forced  to  embark  upon  a  self-organizing  program  of  its  own*- 
and  so  on  down  the  line.  Models  such  as  these  are  quite 
complex,  and  will  require  a  computer  with  a  large  memory  to 
calculate  the  many  contraction  factors  at  the  different  levels. 
In  addition,  "up-and-down"  mappings  (mentioned  briefly  in 
section  2.U)  might  need  to  be  introduced  for  mLnG  systems  with 
m>2,  since  the  mid-level  units  will  not  be  able  to  determine 
(Tf  )e  by  a  procedure  such  as  the  one  described  in  section 
5.2.  For,  these  units  will  normally  not  have  complete  know¬ 
ledge  of  the  system. 
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It  would  be  interesting  to  study  some  special 
cases  involving  non-linear  causal  subsystems.  Reticulation 
of  multi-variable  non-linear  systems  will  most  certainly  have 
to  be  studied  numerically.  Convergence  to  equilibrium  in  the 
manner  that  we  have  used  it  would  probably  occur  only  in 
certain  regions,  if  at  all. 

Needless  to  say,  we  have  only  scratched  the  sur¬ 
face  of  the  theory  of  multi-level,  multi-goal  and  self¬ 


organizing  systems 
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APPENDIX  A 


EXPRESSION  OF  x(k)  IN  TERMS  OF  THE  INITIAL  STATES 
AND  SUBSEQUENT  CONTROL  ACTIONS 

Given  the  system  S  obeying 
x(t+l)  =  Ax(t)+z(t+l)+m(t+l),  t=0,  1,  . T-l> 
x(0)tc, 

where  z(l),  z(2),  . ««,  z(T)  is  a  known  sequence  of  vectors,  we 
can  write 

x(l)  =  Ac+z(l)+m(l)  (A.l) 

x(2)  =  Ax(l)+z(2)+m(2) 

=  A2c+A^z(  l)+m(  1 )]  +z(  2  )+m(  2  ) 

Assume 

x(k)  =  Akc+  Ak”^jjs.(j)+m(j)j  (A. 2) 


Then 

x(k+l)  =  Ax(k)+z_(k+l)+m(k+l) 

=  Ak+W  p  Ak"j fz( j )+m( j )j  +z(k+l)+m(k+l) 

k+1 

=  Ak+1c+  ^  Ak+1”j  [£(j)+m(j)|  | 

hence,  (A. 2)  holds  by  induction  (it  is  true  for  k=l  from 
(A.l)  ). 
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APPENDIX  B 

DIFFERENTIATION  OF  QUADRATIC  AND  LINEAR  FORMS 
WITH  RESPECT  TO  VECTORS 


Given  the  quadratic  form 
n 

g(m)  =  m'Am  =  TZ  a  mm, 

i,j=l  10  1  3 


where 


m  = 


m2 


m 


and  A  is  a  symmetric  matrix,  we  make  the  following  definition; 


dg  = 

3m 


»g 

#*  2 


ag 

am 


thus, 


dE  = 

35 


2U  a«ni 


2H>  a.  m 
j=l  3  3 


*U  *ai*3 


-  2Am. 


Also,  for  f(m)=c'ra=  £Z  c.m.,  the  same  definition  yields 
“  i=l  1  1 


I 


10? 


df  _ 

3m 

For  more  complicated  forms  of  the  type, 
g(m)  =  (Am+c) !  (Am+c) 

=  m'A,Am+£lA»Vm,A,£+£i  c 

we  notice  that  £'Am  =  m'A'c,  so 

g(m)  =  mfA'AnM^tAm+c'c  (B.l) 

and,  using  the  above  derivations, 


(B.2) 


||  =  2A'Am+2A>£  =  2a  *  (AnH-c ) 
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APPENDIX  C 

THE  DYNAMIC  PROGRAMMING  SOLUTION  OF  THE  CONTROL  PROBLEM 

OF  CHAPTER  III 


In  Chapter  ITT  we  encounter  the  following  problem*' 


••minimize 

T  T 

g(M,X)  =  E[i(t)-u(t)| 1  (x(t)-u(tj|+  Y2  m'(t)Dm(t)s 


t=l 


t=l 


subject  to  the  constraint 

x(t+l)  =  Ax(t)+m(t+l)+£(t+l), 
for  t=0,  1,  T-l,  with  initial  conditions  x^0)^*" 

The  vector  sequences  u(l),  u(2),  . ..,  u(T)  and  z(l),  £(2) 
z(T)  are  deterministic,  and  the  matrix  D  is  positive 
definite  and  diagonal. 


(c.i) 

(C.2) 


Because  of  the  Markovian  property  of  the  system 
(C.2)  and  the  loss  function  (C.I)  we  can  app3y  Bellman's 
principle  of  optimality  to  get 

F(N+1,  c)  =  min  (  g-u(T-N~lf] '  [c-u(T-N-l)] 

ml 

+m'Dm^F[N,Ac+ro^z(T-Nj]j-  ,  (C*3) 

where  F(N,c)  denotes  the  minimum  possible  contribution  to  the 
loss  function  if  the  system  is  in  state  c  and  N  decisions  or 
selections  of  m(t)  remain  in  the  process. 

Assume  F(N,c)  can  be  expressed  in  the  following 
form  (note*  »  denotes  transposition): 
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F(N,o)  =  c»R(N)c-2P?(n)c+v(N)  (C.U) 

■where  R(N)  is  a  positive  definite  matrix,  P(N)  is  a  vector, 
and  V(N)  is  a  scalar,  in  particular, 

F(0,c)  =  [|-u(TT]  « jc-u(TjJ 

=  c'c-2u*  (T)c+u» (T)u(T)  (C.5) 

so  that 

R(0)=I,  P(0)=u(T),  V(0)=u»(T)u(T)  .  (C.6) 

From  (C«3)  and  (C«U), 

F(N+l,c)  =  min[  c»c-2u»  (T-N-l)c+u»  (T-N-l)u(T-N-l) 

i  l 


+m» Dm+  jlc+z  ( T  )+m]  »R(N)[ac+z(T-N)4^| 
-2P*  (N )  (Ac+mf z(T-N )J+V(N)  j  . 


(Co  7) 


0 


so 


To  find  the  stationary  point  of  the  expression  in 
differentiate  it  and  set  it  equal  to  0; 


2Dnrt2R(N)m4.2R(N)[Ac4.z(T-^y]-2P(N)  =0 


(C.8) 


m  =  (D4R(N)]”1|p(N)-R(N)[Ac+z(T-n2|  • 

Substitution  of  m  as  given  by  (C«8)  yields,  after  a  few 
algebraic  manipulations, 

F(N+l,c)  =  c’|l+A»  {R(N)“R(N)[T>m(N)]“1R(Ny]Aj'  £ 

-2ju(T-«-l)+A»  [T-R(n')  [IMRCN)]  '3]  [p(N)-R(N)z(T-N)]J  *c 
+u»  (T-N-l)u(T-N-l)-P>  (N )  (d*R (N  fj  jP(N ) 

-2b»  (T-»)  |^r-«(N){p+R(Nif"1J  P(N) 
+z'(T-N)(l-R(N)[&4«(N)]“1j  R(N)z(T-N)+V(N)  (C.9) 
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New,  repeating  the  assumption  made  to  get  (C«U)  for  N+l 
instead  of  N, 

F(N+1, c)  =  c<R(N+l)c-2P' (N+l)c+V(N+l)  .  (C.10) 


Comparison  of  (C.10)  and  (C.9)  yields  the  recursion  relations 


R(N+1) 

P(N+1) 


(C.ll) 


=  I+A'  ^I-R(N)(D4R(NTl"1j  R(N)A 

=  u(T-N-l)+A>  jl-R(N)jD4R(Nf|_1|  [|(N)-R(N)z(T-N)] 

(C  •  12  ) 


V (N+l)  =  u*  (T-N-l)u(T-JN-l)-P»  (N)  (D^CnTJ^CN) 

-2z(T-N)^  I-R(N)[5:m(NTJ''1|  P(N)+z'(T-N) 

n)]“1JrCnMt-n)+v(n)  (c.i3) 

The  method  of  obtaining  the  optimal  policy  is: 


(1)  from  (C.6),  R(0)=T,  P(0)=u(T). 

(2)  calculate  R(N)  and  P(N)  for  N=l,  2,  . .., 
T-l  using  the  starting  values  obtained  in 
(1)  and  the  recursion  relations  (C.ll)  and 
(C.12). 

(3)  From  (C.8),  we  see  the  optimal  policy  is, 
for  N=r-1,  T-2,  . ..,  0, 

m(T -^ ) = [D+R (N 7j  "^P (N ) -R (N )  [Ax(T -N-l) 

+z(T“Nm  , 

since  (C,8)/fis  good  for  any  value  of  the 
state  vector,  in  particular,  the  one  re¬ 
sulting  from  applying  optimal  control  up 
to  that  point. 


The  minimum  value  of  the  loss  function  can  be 
found  ty  carrying  along  V(N)  in  the  recursive  procedure  in 
step  (2)  above.  This  is  simply 


Ill 

gmin  =  m'(1)Dm(1)+F[Tll,  Ac+z(l)4ra(fy] 

=  m* (l)Dm(l) 

+ {ac+z(  l)+m(  lJJ  'R(T-l)  [a  c+z  ( 1  )+m(  1)J 

-2P»  (T-l)[Ac+z(l)+ra(iyj-W(T-l)  (C.ll) 

(U) 


The  above  arguments  are  similar  to  those  used  by  Adorno. 
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APPENDIX  D 

EFFECT  OF  VARIATION  OF  Dn*  AND  D1?*  ON  ( |  IIi|  | 


Let  A  be  given  by  (6«7)>  and  let 


for  i=l,  2,  and  T=3*  Figure  D-l  shows  the  results  of 
evaluating  |jll^||  and  |  | H2 1 1  as  a  function  of  d,  when 


Xi(t)  = 


X  (t) 

' 

Xx(t) 

xjt)  - 

*$(t) 

C. 

x,(t) 

• — 

5 

Figure  D-l. 


1 1  Hill 
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APPENDIX  E 

THE  DYNAMIC  PROGRAMMING  EQUIVALENT  OF  (£.7) 

From  (C.8),  in  order  for  G  to  choose  its  portion 
of  the  optimal  over-all  policy,  we  must  have 


JjCHR (N )J ~1  (p (N ) -R (N )  (Ac+z(T^j  ± 


=  [Dli*4Ri(N)]"1j£i(N)==Ri(N)[Aci+Tri(T-N)]j  (B.l) 

where,  in  the  left  hand  term,  the  subscript  i  denotes  that 

these  are  the  elements  of  the  optimal  controlled  input  vector 

corresponding  to  the  state  variables  under  G^'s  cognizance. 

6  6  6 

G£  has  determined  (WiV  i'e'  %  (U.  JSi  (2)»  •••»  St  (T)  ^ 
the  manner  indicated  in  section  5>.2f,  thus,  from  (Eol)  we 
can  solve  for  P.(N),  and  obtain  a  formula  for  u  *(T-N)  from 
(C.12),  with  appropriate  subscripts  and  z(T-N)  replaced  by 
w^CT-N). 


